Google Research has introduced Nested Learning, a new machine-learning paradigm that reframes a single neural network as a system of interconnected, multi-level optimization problems rather than a monolithic model. In a NeurIPS 2025 paper titled 'Nested Learning: The Illusion of Deep Learning Architectures,' authors Ali Behrouz, Meisam Razaviyayn, Peilin Zhong and Vahab Mirrokni argue that architecture and optimizer are two faces of the same object — different levels of nested learning updated at different frequencies — and that this view opens a new axis for designing continual-learning systems.
Why catastrophic forgetting is the wall LLMs keep hitting
Today's large language models freeze after pre-training. Attempts to keep updating parameters with new data typically trigger catastrophic forgetting, where fresh information overwrites previously learned skills. Nested Learning models the training process itself — including backpropagation — as an associative-memory module. Under that lens, the attention mechanism in Transformers and momentum-based optimizers such as Adam become instances of the same primitive: linear layers that read from a context flow and update at some frequency.
The Hope architecture and a Continuum Memory System
As a proof of concept, the team built Hope, a self-modifying variant of the Titans long-term memory architecture that Google introduced earlier this year. Hope replaces Titans' fixed two-level parameter update with 'unbounded' levels of in-context learning and pairs them with a Continuum Memory System (CMS) — a spectrum of memory modules updating at different frequencies rather than the traditional short-term/long-term split. The result, Google says, is deeper computational depth per token and richer memory management for extended contexts.
Beating Transformers, Titans, Samba and Mamba2 in benchmarks
Google reports that Hope achieves lower perplexity and higher accuracy than modern recurrent baselines including Titans, Samba and standard Transformers on public language-modeling and common-sense reasoning suites. On long-context Needle-in-a-Haystack tasks — using pass-key, number and word probes at three difficulty levels — Hope and Titans consistently outperform TTT and Mamba2, with the CMS blocks credited for the long-context gains.
What it means for foundation models
If Nested Learning transitions from research paper to production, the payoff would be foundation models that can absorb new knowledge post-training without a full retrain — a challenge that has driven the industry toward workarounds like RAG, LoRA adapters and constitutional fine-tuning. Google's post lands alongside a raft of physical-AI world models such as NVIDIA Cosmos Reason 2 and DeepMind's Gemini Robotics 2, where continual learning in the loop is essential for robots operating in changing environments.
The paper's authors credited support from Google Research colleagues Praneeth Kacham, Corinna Cortes, Yuan Deng and Zeman Li. The full technical results and the Hope reference implementation are available in the NeurIPS 2025 paper linked from the Google Research blog.
Reporting based on coverage from the Google Research blog, MarkTechPost, KDnuggets and AI Papers Academy.