Textual Memorilla: How Continua Turns Stanford's Latent Memory Idea into Production-Grade AI Memory

Why "memory" is the bottleneck for serious AI work

Every analyst who has used a chat assistant on a real matter knows the pattern. You upload the file pile, get a confident answer, come back next week — and the assistant has forgotten the amendment you flagged, the add-back policy your credit committee follows, and the conclusion you reversed on Tuesday.

The industry's two default fixes both break down at scale:

1. Stuff everything into context. Expensive, slow, and the detail that matters gets buried. (We wrote about why a bigger upload box isn't the answer in why chat file-upload limits break long-document work.)
2. Retrieve the top-k chunks (RAG). Cheap, but blind to anything that doesn't look like the question. The buried carve-out three amendments deep is exactly what top-k misses.

In late 2026, a Stanford-led team published a third option. We think it's one of the most important memory papers of the year — and it shaped how we built memory in Continua.

What Stanford's MEMORILLA actually does

MEMORILLA: Latent Semantic Memory for LLMs (Fallahpour, Idehpour, Kothapalli, et al., with Jure Leskovec; paper, code) separates remembering from recalling:

• Remember: a frozen encoder stores each incoming document — a conversation, a book chapter — as an embedding, once.
• Recall: for every question, a small set of learned memory slots attends over the question and the whole store, then becomes a handful of memory tokens (16 in their supervised experiments) that a frozen LLM reads alongside the question.

The design rests on four requirements:

The results are striking. Across eight benchmarks spanning personalization, knowledge-intensive QA, and long-document QA, MEMORILLA posted the highest mean score while feeding the model roughly 394× fewer document tokens than full context on average. On NarrativeQA, compression exceeded 4,000×, and with 512 documents end-to-end latency dropped from about 20 seconds to under one.

The most interesting finding for us, though, was qualitative: MEMORILLA wins when an answer depends on how a collection fits together — connecting an iron-deficiency diagnosis from months ago to a question about energy-boosting foods, or verifying a multi-hop claim where the linking entity is never named. On FactKG, as claims composed more facts, retrieval accuracy collapsed while MEMORILLA held above 80%.

That is exactly the problem our customers face in due diligence, commercial credit review, and LME analysis: the clause that changes the outcome rarely shares vocabulary with the question.

The limits the Stanford team names honestly

The MEMORILLA authors are refreshingly direct about what latent memory can't yet do:

• Fine details get lost. A fixed handful of memory tokens can drop exact names — and retrieval still wins on "who" and "which" questions whose answers appear verbatim.
• It needs open weights. Memory is written straight into the LLM's input embeddings, so closed frontier models are out.
• One memory module per model. Each LLM needs its own trained module; transfer across model families is untested.
• Text-only, single-family experiments. All results pair a Qwen encoder with a Qwen decoder.
• Memory is opaque. (Our addition.) You can't open a memory token and read what the system "believes," correct it, or cite it to a reviewer.

For a research prototype, these are reasonable trade-offs. For a lender filing a credit memo or a litigator preparing a deposition summary, every one of them is disqualifying. A memory you can't cite, audit, or correct is a liability in regulated work.

So we asked: what if we kept MEMORILLA's architecture but made the memory itself text?

Introducing Textual Memorilla

Textual Memorilla is Continua's memory design. It maps MEMORILLA's remember/recall split and all four of its requirements onto a durable, human-readable substrate: the Continua Project Room.

In short: MEMORILLA compresses into vectors; Textual Memorilla compresses into evidence.

How Textual Memorilla works, step by step

1. Remember: compile once, version forever

When a file enters a Project Room, PDFs, Word, Excel, PowerPoint, email, and other binaries are compiled into line-numbered Markdown, and text formats such as Markdown, JSON, and CSV are available the instant they land. Every write publishes a new immutable version with its own file ID — the room is built on multi-version concurrency control (MVCC), the same technique databases use to let many readers and writers share state without locks.

This is MEMORILLA's "reusable document embeddings" requirement, with two production upgrades: a snapshot read always resolves to exactly one version (so reasoning is reproducible), and source dates travel with each file (so "which figure is current?" has an answer).

2. Recall, part one: search to orient

MEMORILLA's own analysis shows retrieval still wins for verbatim lookups. So Textual Memorilla keeps it — as a cheap, scoped search primitive that returns ranked passages in seconds, each with file ID, line numbers, and source date. Agents use it to get their bearings without spending their context budget.

3. Recall, part two: delegate to decide

When an answer must hold against everything, the agent hands the question to the complete-record engine in one call. The engine sweeps every file — across several rooms, plus web search and MCP-connected systems if you choose — and returns verified findings. The record never enters the agent's context; only the conclusions do.

This is the textual analogue of MEMORILLA's memory slots: the whole collection is read, the question shapes what comes back, and the downstream model receives a compact, fixed-shape result.

It is also the hybrid the Stanford team flags as a promising next step. Their oracle analysis found that on FactKG, about 16% of claims were solved only by learned memory and 10% only by retrieval. Search + delegate is designed to capture both sides of that union.

4. Consolidate: memory that compounds

Here Textual Memorilla goes beyond what MEMORILLA attempts. MEMORILLA reads a store; it doesn't curate one. Continua distills the task chain itself into living knowledge files — not just answers, but follow-ups, expert corrections, business decisions, and conclusions that were later reversed:

Enterprise Knowledge Base/
├── knowledge-index.md
├── domain-knowledge-<topic>.md      # what is true, and since when
├── work-methods-<topic>.md          # how to handle a problem
├── decision-cases-<topic>.md        # context, options, rationale, outcome
└── evidence-gaps-and-conflicts.md   # what the record can't yet resolve

The engineering choices matter more than the folder:

• Incremental, not rewritten. Each run compares sources against the last successful processing record and handles new, changed, unchanged, and unfinished items differently. Memory that's re-summarized from scratch every week quietly forgets.
• Fixed filenames, updated in place. New evidence can add to an entry, correct it, merge a duplicate, or narrow where it applies — not just append.
• Verified writes. A source is marked processed only after its knowledge is written and read back.
• Never learns from itself. The knowledge base's own files are excluded as sources, so outputs aren't re-distilled into echoes.
• Gaps are first-class. Conflicts and missing evidence live in their own file until the record resolves them.
• Every entry points back to its source room, file or task, location, and date.

This is the layer that powers the Enterprise Second Brain, and it's shared: team rooms let every analyst and every agent read the same knowledge, policies, and reusable skills.

5. Act: memory for long-running agents

MEMORILLA's TextWorld experiment showed that eight learned memory tokens help an agent stop looping and finish more games. We took the same lesson — agents need memory of their own experience — and engineered it for unattended work that can run for hours or days.

In Continua's self-driving agent loop, the goal, acceptance criteria, plan, memory, and budget are ordinary room files. Each round:

1. reads state and the write-ahead log,
2. checks budget and claims one plan item with a checkable "done when,"
3. reasons and executes (pausing for a person before anything dangerous and irreversible),
4. writes state back as a new version,
5. schedules the next round with an idempotent key, so retries never spawn duplicates.

An interrupted run is recovered, not repeated. And because writes are MVCC-versioned, a reviewer can edit the plan mid-run while a scheduled wake-up and a sub-analysis write to the same room — conflicts surface as a clean HTTP 409 instead of silent corruption.

Latent vs. textual memory: when each wins

We don't think latent memory is a dead end. It's a genuinely better fit for some workloads:

• Latent memory (MEMORILLA-style) is attractive for high-volume, low-latency personalization on a single open-weights model, where sub-second answers matter more than citations and occasional loss of an exact name is acceptable.
• Textual memory (Continua) is the right default when findings will be relied on, filed, or presented — credit committees, legal teams, regulators — where every claim must be cited, current, correctable, and owned by your team rather than locked inside one model's embedding space.

It also has a privacy property that MEMORILLA's authors themselves highlight as desirable: memory you can see is memory you can delete. Continua runs with zero document retention for normal tasks, and your documents are never used to train AI models — improvement happens through execution traces and human-approved knowledge files, not model weights.

What this looks like in practice

A credit team reviewing Borrower B points one investigation at three rooms: the deal room, the team's credit policy, and last year's precedents. The agent searches to find the P&L and the add-back schedule, then delegates the hard question — do any add-backs conflict with policy? — to the complete-record engine.

The finding comes back cited to both sides: the add-back in Borrower B/P&L.xlsx and the rule at Policy/addbacks.md:41. When the analyst overrides one classification, that correction flows into decision-cases-addbacks.md with its source and date. The tenth review starts where the ninth left off.

That's Textual Memorilla: whole-collection recall, a fixed budget into the model, and memory your team can read. Try the same pattern on a single check with our free EBITDA Add-back Analyzer or Financial Covenant Extractor.

Frequently asked questions

What is Textual Memorilla? Textual Memorilla is Continua AI's memory architecture. It adopts the core design of Stanford's MEMORILLA — read the entire collection, recall only what the question needs, keep a fixed budget for what reaches the model — but stores memory as versioned, cited Markdown in Project Rooms instead of latent memory tokens.

Is Continua affiliated with the Stanford MEMORILLA team? No. MEMORILLA is independent academic research from Stanford and collaborating institutions. Textual Memorilla is Continua's engineering adaptation of the ideas the paper publishes.

How is this different from RAG? RAG passes the top-k most similar chunks to a model and never sees the rest. Textual Memorilla uses retrieval only to orient, then delegates to an engine that reads 100% of the record. See Continua vs. retrieval-based AI and our benchmarks, where unsupported claims per report fell from 6.8 to 0.4 on the same model weights.

Does Textual Memorilla require a specific model? No. Because memory is text, any model can read it. Continua routes each step across 20+ models, or you can pin the model your policy requires per task, project, or API call.

Can my own agents use it? Yes. The Continua MCP server and API expose rooms, files, versions, search, tasks, and schedules, so your agents can read team memory and write results back.

Who controls what the system remembers? You do. Knowledge files are plain, versioned room files your team can read, edit, and restore. Consequential actions wait for human approval, and every change is in the audit trail. Read more on Trust & Security.

Give your AI a memory you can cite

MEMORILLA proved that reading everything and recalling only what matters beats both brute-force context and top-k retrieval. Textual Memorilla makes that idea ready for work you have to defend.

Create your first Project Room — free · See how the platform works · Compare plans

Further reading: Continua vs ChatGPT · Continua vs Claude · Continua vs Gemini Notebook · Build a knowledge graph · Find contradictions across documents

Source: Fallahpour, A., Idehpour, P., Kothapalli, V., Mounier, N., Wu, S., Pardis, S., & Leskovec, J. (2026). MEMORILLA: Latent Semantic Memory for LLMs. COLM Workshop on Context Beyond the Window. memory.stanford.edu. MEMORILLA figures are as reported by the authors.