6.1 Sequence and Blocking

\(S = B_1 \oplus B_2 \oplus \cdots \oplus B_m\), where block \(i\) is the token span

\[ B_i = x_{s_i : e_i} \]

\(s_i\): global start index of block \(B_i\) (inclusive);
\(e_i\): end index (exclusive, half-open interval, Python slice semantics).
Example: \(s_1=0,\, e_1=384\) means \(B_1\) occupies positions \(0,1,\ldots,383\).

6.2 Mixed Cache \(L\)

For each global position \(t\) and layer \(\ell\):

\[ L[t,\ell] = \bigl(\hat K_{t,\ell},\; \hat V_{t,\ell},\; m_t\bigr) \]

• \(\hat K_{t,\ell}, \hat V_{t,\ell}\): KV after load and concatenation;
• \(m_t = (\type_t,\, b_t,\, g_t)\):
  • \(\type_t \in \{\mathrm{global},\mathrm{local}\}\) — whether KV at \(t\) matches the full-prefill oracle;
  • \(b_t\) — block id;
  • \(g_t\) — global position (used by RoPE and \(F\)).

Compact notation (illustrative, not a formal definition):

\[ L \;\approx\; [\;\underbrace{K^{\mathrm{global}}V^{\mathrm{global}}}_{B_1\;\text{global KV}},\;\underbrace{K^{\mathrm{local}}V^{\mathrm{local}}}_{B_2\;\text{local KV at load}},\;\ldots\;] \]

Superscripts global / local denote KV quality type; subscripts denote block id—blocks are not fixed to one type: \(B_1\) is usually all global under a true-prefix assumption; \(B_2\) is often local at load and may become global after \(F\) write-back on \(\mathcal{T}_\tau\).

6.3 Recomputation Set \(\mathcal{T}_\tau\)

Symmetric window at seam \(B_i \mid B_{i+1}\):

\[ \mathcal{T}^{(i)}_{\mathrm{seam}} = [\,s_i,\, e_i)\cap[e_i-\tau,\,e_i) \;\cup\; [\,s_{i+1},\, e_{i+1})\cap[s_{i+1},\, s_{i+1}+\tau) \]

Asymmetric rule (true prefix \(B_1\)): at seam \(B_1|B_2\), skip the trailing \(\tau\) tokens of \(B_1\); reallocate that budget to the trailing \(\tau\) tokens of \(B_m\).

Recomputation ratio \(|\mathcal{T}_\tau|/N\); engineering target: \(\le 5\%\) while meeting accuracy.

6.4 Functional Form of Operator \(F(Q,L)\)

\(F\) is a structure-aware extension of standard self-attention: given query set \(Q\) and mixed cache \(L\), it outputs hidden states aligned with the oracle:

\[ F(Q,\,L) \;=\; \Attn\!\bigl(Q,\, \hat K,\, \hat V;\, \mathcal{M}_{\mathrm{corr}}(L)\bigr) \;\approx\; \Attn(Q,\, K^{\star},\, V^{\star}) \]

where \(\mathcal{M}_{\mathrm{corr}}(L)\) is the correction mask derived from metadata in \(L\):

\[ \mathcal{M}_{\mathrm{corr}}(L) = \{M_{\mathrm{local}}(j),\; M_{\mathrm{sink}}(j)\} \]

\[ M_{\mathrm{local}}(j)=\mathbb{1}[\type_j=\mathrm{local}], \qquad M_{\mathrm{sink}}(j)=\mathbb{1}[j\in\mathcal{S}'] \]

• Prefill · physical repair mode: for \(t\in\mathcal{T}_\tau\), \(F\) runs full layered causal forward, writes back \((\hat K_{t,\ell},\hat V_{t,\ell})\), and sets \(\type_t\leftarrow\mathrm{global}\);
• Prefill / decode · inference mode: attention forward only (no write-back), using \(\mathcal{M}_{\mathrm{corr}}(L)\) to correct \(E_3\);
• Degeneration: when \(\tau \ge |B|/2\) with asymmetric coverage of \(B_{2:m}\), \(L\approx L^{\star}\), \(\mathcal{M}_{\mathrm{corr}}(L)=\varnothing\), and \(F\) equals standard \(\mathrm{Softmax}(QK^{\top}/\sqrt{d})V\).

\(L^{\star}\): oracle cache from one full prefill over \(S\) (reference baseline, not required online).

Logit form (single layer)

\[ s_{ij} = \frac{\langle q_i,\,\hat k_j\rangle}{\sqrt{d}} + \underbrace{\alpha\,\mathbb{1}[\type_j=\mathrm{local}]}_{\text{local KV tilt}} + \underbrace{\beta\,\mathbb{1}[j\in\mathcal{S}']}_{\text{spurious sink penalty}} \]

\[ o_i = \sum_j \mathrm{softmax}_j(s_{ij})\,\hat v_j \]

• \(\alpha < 0\) (or equivalent up-weight on global keys): steer mass away from unrepaired local KV;
• \(\beta < 0\): penalize per-block leading tokens \(\mathcal{S}'\) except the first \(s\) true sinks at sequence start;
• after step ③ write-back, more \(\type_j=\mathrm{global}\) → \(\alpha\) term weakens automatically.

Degeneration of \(F\) (empty correction terms, not a separate regime)

Correction mask set (consistent with §6.4):

\[ \mathcal{M}_{\mathrm{corr}}(L) = \{M_{\mathrm{local}}(j),\;M_{\mathrm{sink}}(j)\}, \;\; M_{\mathrm{local}}(j)=\mathbb{1}[\type_j=\mathrm{local}], \;\; M_{\mathrm{sink}}(j)=\mathbb{1}[j\in\mathcal{S}'] \]

When \(\mathcal{T}_\tau\) is large enough (e.g. \(\tau\ge |B|/2\) with asymmetric coverage of \(B_{2:m}\)):

• step ③ has written back global KV → no positions with \(\type_j=\mathrm{local}\);
• spurious sinks removed by penalty and write-back → \(\mathcal{S}'\) effectively empty;
• hence \(\alpha\) and \(\beta\) terms are zero maps on empty sets,

\[ F(Q,L) \;\equiv\; \mathrm{Softmax}\!\left(\frac{QK^{\top}}{\sqrt{d}}\right)V \]

i.e. automatic degeneration to standard Transformer self-attention. Computation still runs full causal prefill on the suffix; reuse benefit mainly retains \(B_1\) cache hits.