MemT-1.0 Released to Keep Conversations Inside the Model, 1.86x Faster on Long Histories

MemTensor researchers released the technical report for MemT-1.0 on October 19 local time, describing a system that stores conversations inside the model itself. Two models were released alongside it: Nano and Flash. The numerical values that make up the models (the weights) and the code needed to run them were also published, so anyone can download them and try them out. The core idea is that instead of feeding the full text of past turns back in every time, the information gleaned from a conversation is held in the model’s internal memory.

How does it remember past conversations?

MemT-1.0 reads from its internal memory when generating a response. With the memory-writing function turned on, it folds information from both the current turn and any earlier turns not yet stored into memory once the answer is finished. The memory state for each conversation can be saved and reloaded later. The design is meant to let information accumulated in earlier turns carry forward.

The work builds on the earlier Metis research. Unlike approaches that carry the entire conversation history along or search external documents for the relevant passages, this one holds the information in the model’s internal state. It is an attempt to reduce the burden of continuously storing and re-reading raw text when handling long conversations.

How much faster is it on long conversations?

The researchers report that Nano improved on both processing speed and storage footprint for long conversations. Under conditions where the conversation history reached 128,000 tokens, end-to-end processing from preparing the history to producing an answer was about 1.86 times as fast. The space required to keep the memory on hand shrank to roughly one sixty-sixth. The comparison baseline is the widely used approach of caching the computed results of the conversation history as-is.

It does not come out ahead on every task, however. Across the 11 memory benchmarks the researchers ran, the two models placed first and second among approaches that hold memory inside the model. On tasks that require pinpointing exact supporting evidence in a long history, by contrast, approaches that write memory down as text performed better.

What do you need to use it?

For now, the only way to use it is to download the models and run them yourself. The official documentation assumes a Linux server with Nvidia GPUs. This is not a chat app for general users; it is at the stage where developers and researchers with their own servers who can run the models directly will be the first to take a look.

Companies considering it for business use need to check the terms for the code and the models separately. The inference software in the public repository carries terms that prohibit commercial use. The Flash model weights, on the other hand, are marked with the MIT license, which permits free use. Even within the same project, the terms differ.

MemT-1.0 moves conversational memory into the model’s computation, easing the burden of processing long histories. What remains is to preserve that speed while also raising the model’s ability to locate precise supporting evidence. For users in Korea, it is worth weighing memory accuracy in Korean-language conversations alongside the available routes for using it domestically and operating costs.