Events can be found on calendar page, with code AC.
An informal seminar to connect researchers in Africa, China and elsewhere.
Focussing on pure mathematics and related topics.
Welcome to register for updates: from China / elsewhere.
Organizer: Will Donovan
We plan to use the following format, to promote conversation in online setting.
Nov 28 2024
On the Largest Prime Factor of the k-generalized Lucas Numbers
Dec 5 2024
Exploration of Cyclotomic Extensions
Mar 5 2025
Phase Transitions in Large Language Models and the O(N) Model
Large language models (LLMs) exhibit unprecedentedly rich scaling behaviors. In physics, scaling behavior is closely related to phase transitions, critical phenomena, and field theory. To investigate the phase transition phenomena in LLMs, we reformulated the Transformer architecture as an O(N) model. Our study reveals two distinct phase transitions corresponding to the temperature used in text generation and the model’s parameter size, respectively. The first phase transition enables us to estimate the internal dimension of the model, while the second phase transition is of higher-depth and signals the emergence of new capabilities. As an application, the energy of the O(N) model can be used to evaluate whether an LLM’s parameters are sufficient to learn the training data.
Mar 12 2025