Mathematical Theory of the Transformer : Foundations, Rationality, and the Incremental Transformer

Omtale

The Transformer has reshaped machine learning, yet its now-canonical architecture was assembled empirically rather than derived from first principles. Its components—content-based attention, residual connections, layer normalisation, positional signals, and feed-forward networks—were justified by benchmark performance, not by a theory of why the whole should work. This book provides that missing theory, developing a mathematical account in which architectural choices cease to be independent hyperparameters and become derived objects tied to a small number of geometric and dynamical quantities that can be measured, diagnosed, and controlled. Key Features: Rigorous mathematical foundations: Provides a comprehensive mathematical theory of why the Transformer architecture works, transforming empirical design into derived principles grounded in differential geometry, Lie algebra, and dynamical systems. Constructive diagnostic framework: Introduces practical tools—antisymmetry indices and a four-class taxonomy of attention heads, an eight-regime classification, redundancy measurement—that enable principled architecture design, yielding the Rationalised and Incremental Transformers, with measured efficiency gains and their limits reported. Transparent scholarly methodology: Every claim is explicitly labelled as a theorem, empirical finding, conjecture, or design principle, with a complete proof for every theorem given in the text or in the Online Supplement, setting a new standard for credibility in the deep learning literature. Written for graduate students and researchers in machine learning, applied mathematics, and theoretical computer science, this book serves as a primary or supplementary text for postgraduate courses on deep learning theory and neural architecture design. It is equally valuable as a reference for practitioners working in mechanistic interpretability, model compression, efficient attention, and energy-aware AI systems.