memt-1.0 Audited. When Memory in Weights Beats Storage

memt-1.0 asks an uncomfortable question of anyone who just designed an agent memory stack. If the model carries memory in its own weights and recurrent state, why did you build the vector store, the reranker tier, and the KV cache hierarchy at all? The release reports a 66x reduction in persistent agent memory footprint against a full-context KV cache baseline, and that number is currently doing a lot of persuasion work in architecture reviews. This piece audits it. The conclusion up front is that memt-1.0 relocates agent memory's binding constraint rather than deleting the storage hierarchy, moving it from inference-time bandwidth to training-time capacity and update economics. Once you price that relocation honestly, the fork between memory in weights and memory in storage resolves on three variables you can estimate before writing code: agent lifetime, data mutation rate, and compliance exposure.

Three questions drive what follows. Is the 66x a property of the system or an artifact of the baseline? What does parametric memory actually cost to update and delete? And does the external hierarchy you were told to build survive contact with a memory-native agent architecture?

What Changes When an Agent Remembers in Its Weights

Nearly every agent memory guide, including the external-storage coverage on this site, starts from the same assumption. The model is stateless, and memory is a bolt-on: episodic logs, embedding indexes, governed tiers of hot and cold data, and MemGPT-style virtual context paging to stretch a fixed window. The stack exists because the transformer forgets between calls, so the engineering goes into retrieval quality, cache placement, and tier governance.

memt-1.0 attacks the assumption rather than tuning the stack. Its memory-native design internalizes state into the computation layer, so recurring procedures and a user's stable preferences live in model parameters and recurrent state instead of in rows you fetch each turn. The project describes continuously internalizing retrieved procedural experience during multi-turn tasks, with token-level memory updates inside each turn. That inverts the usual division of labor. The external store becomes a capacity supplement rather than the substrate of memory itself.

The inversion matters because it changes which resource you exhaust first. A stateless model plus store spends attention and bandwidth re-reading context every turn. A memory-native model spends parameter capacity and training compute on consolidation. The rest of this article is the price list for that swap.

Inside gated deltanet-2 and the Mixture-of-Storage Design

The core mechanism, gated deltanet-2, is a linear attention memory built on a gated delta rule. Delta-rule models maintain an associative memory matrix as recurrent state and update it per token, first removing a key's old association before writing the new value, which suppresses the interference that plagues plain additive fast-weight updates. Gating adds a learned retention factor so stale content decays instead of accumulating forever. The gated delta networks paper formalizes this family, and the property that matters here is architectural: the state is fixed-size, set by state dimensionality rather than by how many tokens or turns the agent has processed.

That single property is the source of every footprint claim. A transformer's KV cache grows linearly with history; a delta-rule state does not grow at all. Baking memory into the computation layer is an old idea, with a fast-weight memory lineage running back through neural Turing machines and differentiable neural computers. The contribution is making it survive a modern LLM and agentic workloads.

Then there is the mixture-of-storage layer, which exists because a fixed-size state has a capacity ceiling. memt-1.0 keeps a bounded external tier to scale capacity instead of pretending the state is unbounded. The system is memory-native with an asterisk, which is exactly what you would design if neither weights nor storage won outright.

Auditing the 66x Footprint Claim

Treat the 66x as a release claim, not a verified benchmark. It comes from the project's own memt-1.0 release materials, measured on memt-1.0-nano at a 128k token history against a full-context KV cache setup, alongside a reported 1.86x end-to-end inference speedup. No independent replication exists yet, so the measurement conditions deserve more attention than the number.

Start with the baseline. Persistent KV for a full 128k context is enormous by construction. For an 8B-class model with 32 layers, 8 KV heads, 128 head dimensions, and fp16 storage, KV state costs about 128 KiB per token, which reaches roughly 16 GiB per session at 128k tokens. Serving research treats KV cache bandwidth costs as a first-order constraint in long-context inference for exactly this reason. Against that baseline, any constant-size state wins by orders of magnitude, so the arithmetic makes 66x plausible while telling you surprisingly little.

Three audit notes follow. First, the ratio is a function of history length, not a property of the system. Constant state against linearly growing KV means the multiple at 64k is roughly half and at 256k roughly double, and most production agents live well below 128k, where the gap shrinks accordingly. Second, the baseline is close to a worst case that few production stacks actually run. Competent setups avoid persisting full-context KV through paging, eviction, summarization, or retrieval, so the realistic opponent is a compressed external store and the multiple against it will be far smaller. Third, the comparison excludes what the mixture-of-storage tier itself stores and says nothing about recall quality. The more informative number in the release is arguably the 1.86x speedup, since end-to-end latency reflects the workload you would actually run.

None of this says the direction is wrong. Constant-memory agents are an active research line, and MEM1's constant-memory results show the same pressure toward fixed state from a different starting point. It does mean the 66x should prompt your own measurement rather than seed your capacity plan.

Fixed Parametric Capacity vs a Store That Grows

A footprint win can hide a recall ceiling. Parametric memory saturates. A delta-rule state holds a bounded number of key-value associations before interference degrades retrieval, and the bound comes from state dimensionality and parameter count, not from how much your agent needs to know. An external store grows with data volume and keeps recall exact as cardinality climbs. Comparing the two on bytes compares a cache to a disk and declares the cache the winner.

The practical split runs along knowledge type. Procedural patterns, recurring tool chains, and one user's stable preferences compress well into weights because they are low-cardinality and reused constantly. High-cardinality corpora, per-tenant document sets, and append-heavy event logs belong in a store because they exceed any fixed state and need precise recall. When parametric memory versus external storage gets framed as winner-take-all, the mixture-of-storage design is the refutation shipped inside the product.

Retraining Weights vs Hot-Swapping a Store

Update economics is where the designs diverge hardest. An external store gives you CRUD. Writing, correcting, or rolling back a fact is a millisecond-scale, atomic, auditable operation, and cache invalidation is cheap. Memory in weights gives you gradient-based consolidation. Changing one fact means running optimization steps over data containing it and then regression-testing the model, because gradient updates risk damaging adjacent knowledge. Survey work on catastrophic forgetting catalogs how badly naive fine-tuning erodes prior competence, and every consolidation pass carries that risk.

Two asymmetries follow. Singleton updates are terrible for weights and trivial for stores; a changed address should never trigger a training job. Bulk updates amortize better for weights, since consolidating a thousand facts costs roughly one pass while a store pays per row. And weights have no transactional rollback. Once a pass degrades behavior, restoring a checkpoint loses whatever else that pass taught.

memt-1.0's own pipeline is the cost signature in miniature. A two-stage continued pre-training and post-training run over a reported 34B tokens keeps general reasoning competitive while building parametric memory. That is the price of writing to weights at release scale, and it does not shrink to zero because your update is small.

Unlearning and Right to Be Forgotten Inside Weights

Deletion is where the structures genuinely differ. Removing a fact from an external store is a row-level delete, an index purge, and an audit log entry, and you can prove absence to an auditor. Removing a fact from weights is an open machine-unlearning problem. Exact removal of a single fact's influence generally requires retraining from a clean checkpoint or from scratch, and approximate unlearning methods lack strong guarantees, leaving remnants that extraction and membership-inference attacks can still detect.

The gap maps directly onto compliance. Legal analysis of GDPR erasure highlights how erasure obligations collide with models that have absorbed personal data into parameters. If your agent internalizes user PII into weights and a user invokes erasure, the options are approximate unlearning with weak guarantees or expensive retraining, and neither produces clean audit evidence. For compliance-bound workflows in healthcare intake, financial onboarding, or EU-facing personalization, this alone can settle the fork toward external storage regardless of footprint or latency. The store is where deletion remains a first-class operation.

Five Workloads and Where Each Memory Design Wins

Abstract tradeoffs resolve quickly once attached to concrete agents. Five archetypes cover most of the design space.

The pattern is consistent. Short-lived or single-tenant agents with stable knowledge tilt toward weights. Multi-tenant, audit-bound, or unboundedly growing agents tilt toward storage. The crawler row is the honest middle, and it is not an edge case. It is where long-horizon agents actually live.

A Decision Rubric for the Memory Fork

Score it bluntly. If four or more rows land in the weights column, a memory-native design such as memt-1.0 deserves a prototype, and the constant footprint plus faster long-context inference will compound in your favor. If compliance or mutation rate lands in the storage column, weights cannot carry the load alone regardless of the headline number, because you would be pricing out deletion and update economics. Mixed columns describe most real agents, which is why mixture-of-storage exists as a design category rather than a compromise.

What memt-1.0 Means for the Storage Stack You Planned

Close the loops. The 66x is real arithmetic measured against the wrong opponent; it compares constant state to append-only KV stacking, a baseline few production stacks run, and it scales with history length rather than describing a fixed property. Memory in weights buys genuine wins, constant footprint, faster long-context inference, no retrieval round trips, and pays in capacity ceilings, consolidation latency, and unlearning risk. The hierarchy survives, smaller. For most builders the reading is that the KV tier shrinks, procedural memory migrates into weights, and the governed external store keeps everything that must scale, mutate, or be deletable.

That is also the honest update to this site's external-storage thesis. The stateless model was an accident of transformer serving, not a law, and memory-native designs are correcting it. But memt-1.0 shipping a mixture-of-storage layer is the tell that the correction converges on hybrid stacks rather than the deletion of storage engineering. If you were about to build an agent memory architecture, build it, and design it assuming the model above it remembers more of its own history than last year's stack did. Put your differentiation in the parts weights cannot hold, which is precisely the data that grows, changes, and must someday be forgotten.