Linear-time sequence models and structured state-space models (SSMs) achieve O(L) inference scaling but provably suffer from exponential memory decay on associative recall tasks over context lengths exceeding 128k tokens. We derive a spectral-gated hybrid operator that interleaves rank-r Hankel state compressions with sparse local attention heads, achieving O(L log L) complexity with tight lower bounds on long-horizon associative recall.
Diagonal state-space models rely on HiPPO-initialized transition matrices whose eigenvalue spectrum clusters near the unit circle. In v1, Theorem 3.2 was proved only for exact real arithmetic without BF16 rounding.
Initial benchmarks evaluated multi-query needle retrieval up to 256k context length on 4x A100 GPUs.
UPDATED IN V2: In response to Reviewers 1 & 2, we prove Theorem 3.2 under IEEE-754 BF16 quantization, showing that block-diagonal Hankel preconditioning bounds state drift below 1e-4 over 1M steps.
UPDATED IN V2: Per Reviewer 3 request, benchmarks on 8x H200 GPUs demonstrate 4.3x higher throughput than FlashAttention-3 at 1M sequence length with 99.4% exact multi-query needle retrieval.
We thank all three open reviewers for their rigorous mathematical and empirical critique. Side-by-side in v2 ("After Manuscript"), readers can verify: (1) Full finite-precision BF16 proof of Theorem 3.2 in Section 1 & Appendix B; (2) 1M-token needle-in-a-haystack ablations against Mamba-2 and FlashAttention-3 in Section 2; and (3) Open-source Triton/CUDA kernels.
0000-0002-1190-3321 ·
[ORIGINAL REVIEWER · Round 2 on v2]
Public Evaluation: Theorem 3.2 is a landmark theoretical result for hybrid SSM-attention architectures. Comparing the Before (v1) and After (v2) manuscripts confirms that the BF16 quantization proof is completely watertight.
Technical Critique / Verification Checklist: All comments addressed in the v2 After Manuscript.
0000-0003-8821-5543 ·
[ORIGINAL REVIEWER · Round 2 on v2]
Public Evaluation: The 1M-token empirical evaluations and open Triton kernels added in v2 make this immediately reproducible and highly impactful.
Technical Critique / Verification Checklist: Endorsed without reservations following v2 revision.
0000-0001-4490-1287 ·
[FRESH REVIEWER · Round 2 on v2]
Public Evaluation: Cloned the open-source repository and reproduced Figure 4 on a single node. Fully endorse publication as a Peer-Validated Article.
Technical Critique / Verification Checklist: None.