Fourier-Reconstructible Local Attention and Compressive Memory

A problem statement about spectral complexity in localized sequence-model structure.

Active research. This page is intentionally a problem statement rather than a progress report. Current lemmas, conjectures, calculations, experiments, and proof strategies are omitted while the project is active.

Background

Self-attention produces structured matrices describing how information at one position is combined with information elsewhere. Local causal attention restricts that interaction to a moving window. Separately, Fourier-Ratio methods provide a notion of effective spectral complexity that does not require exact sparsity and can, in suitable settings, support recovery from incomplete data.

Problem

Can local attention or memory objects admit a spectral-complexity description that is mathematically stable and algorithmically useful? Localization is the central difficulty: a global Fourier bound does not automatically remain informative after one breaks an object into local pieces.

Tools

There are two pieces of background I need on the page: finite Fourier complexity and the bare definition of local causal attention. I am stopping before the active bridge between them.

Finite Fourier Ratio

Definition. Let \(G\) be a finite abelian group and let \(\widehat f\) be the normalized Fourier transform of \(f:G\to\mathbb C\). For \(f\neq0\), \[ \operatorname{FR}(f)=\frac{\|\widehat f\|_1}{\|\widehat f\|_2}. \]

Proposition. For a coefficient vector of dimension \(D\), \[ 1\leq\operatorname{FR}(f)\leq\sqrt D. \]

Local Causal Attention

Definition. A causal attention rule of width \(W\) is a collection of nonnegative weights \(A_{ij}\) satisfying \[ A_{ij}=0\quad\text{unless}\quad 0\leq i-j

This fixes only the information geometry; it does not assume a particular Transformer parametrization.

Fourier-Ratio Recovery

Theorem (schematic recovery form). If \(\operatorname{FR}(f)\leq r\) on a finite abelian group \(G\), then random sampling on the order of \[ r^2\varepsilon^{-2}\log^2\!\left(\frac r\varepsilon\right)\log|G| \] measurements is sufficient, under the standard hypotheses, for stable Fourier \(\ell^1\)-recovery with \(L^2\) error bounded by a constant multiple of \(\varepsilon\|f\|_2\) with high probability.

Localization Obstruction

Proposition. Suppose \(G=H\oplus K\) and let \(f_k\) denote the slice indexed by \(k\in K\). Then \[ \max_{k\in K}\operatorname{FR}_H(f_k) \geq \frac{\operatorname{FR}_G(f)}{\sqrt{|K|}}. \]

So global spectral simplicity does not automatically become uniform local simplicity. The public page stops here rather than giving the current formalization or recovery construction.

Direction

The project asks what “spectrally compressible attention” should mean and when such a notion would actually buy something. I am not posting the current formalization, hypotheses, reconstruction mechanism, or model experiments while the project is active.

Last updated: September 14, 2026.