Exact Input Writes Improve Stable Looped Language Models

Wed, 29 Apr 2026 12:00:00 -0400

Make Parcae's input write exact.

Same loop, same full-matrix write family; replace only the input gain.

\(h_{t+1}=\exp(-\Delta a)\odot h_t + \text{input write}+\bar{\mathcal R}(h_t,e)\)

Euler input gain

\(\Delta Be\)

replace gain

Exact-ZOH gain

\(\dfrac{1-\exp(-\Delta a)}{a}\odot Be\)

ZOH = zero-order hold: hold \(e\) fixed during one loop step, then integrate the input exactly.

TL;DR: Exact-ZOH replaces Parcae's Euler input gain with the held-input integral for the same full-matrix write. It lowers validation loss in completed matched 140M controls; CORE Extended is slightly higher and CORE is slightly lower on the 11.2B paper-style pair.

Parcae on Husky'Log

Exact Input Writes Improve Stable Looped Language Models