Google Unveils an AI "Sleep" Training Method That Forgets Less as It Learns
The method adds a "sleep" phase that moves information gathered in conversation into long-term memory. Google researchers say it beat existing training approaches on tests of translation, fact learning, and math problem solving.
Google Research unveiled its “sleep framework” on October 27. It is a training approach that sets aside time for an AI to organize and store what it learned, separate from the time it spends talking with people. The researchers said the method reduces the problem of forgetting existing knowledge while picking up new facts, and also improves the model’s ability to solve complex problems.
Why would an AI need sleep?
Because AI has a hard time holding on to what it has just learned. Anything a model reads mid-conversation stops being usable once that conversation ends. But continuously training the model on new data brings on “catastrophic forgetting,” where performance on tasks it previously handled well drops sharply, the researchers explained.
The sleep framework takes its cue from the way the human brain consolidates memories while at rest. The core idea is to move short-term memories acquired during activity into a form that persists inside the model. The researchers proposed a training cycle that alternates between time spent exchanging information with people and time spent organizing memory.
What happens during sleep?
First, memories are filtered. Rather than storing everything it encounters, the model selects and compresses only information that is coherent and useful for its work, the researchers explained. The design also sets aside separate space for new material so that already learned knowledge is not overwritten.
Image: research.google
Next comes the “dreaming” stage. The model generates practice examples on its own, then picks the ones that help it learn and uses them to sharpen its skills. This process takes place only within the training rules and model structure that humans set in advance.
What did the translation and math tests show?
In a translation test where the model learned languages it had never seen before, such as Manchu and Kalamang, one after another, performance barely degraded under the sleep approach, the researchers said. Under the same conditions, methods that rely on conversational context and supervised fine-tuning showed substantial forgetting of what had been learned earlier, they explained.
Image: research.google
On BABILong, a long-context comprehension test, the setup handled information amounting to as much as 10 million tokens. The researchers said the sleep approach, which compresses long material into the model’s internal memory, outperformed GPT-4 and Titans. On evaluations of absorbing new facts, they reported 48.9% when given a single task and 46.2% under continued pretraining.
Scores improved on math problem solving as well. A model using the sleep approach reached accuracy rates of 79.2% on AIME 24, 69.0% on AIME 25, and 46.1% on HMMT 25. The researchers said these beat other training methods, including supervised fine-tuning.
For users, one question matters most: can an AI keep remembering what you told it in conversation while still doing everything it was already good at? Whether the method will be used in real products, and how much time and money memory consolidation will require, remain unknown for now.