Fold, don't forget

In Kylon, an agent is a long-term member of your team's channels. It joins discussions, runs tasks, and stays on the job for months. That creates a problem chatbots never face: the conversation never ends, but the model's context window never grows.

This is how we solved it: by folding history into a tree that can always be unfolded again, quietly, in the background. Along the way: an interactive demo, how memory follows a thread when you fork one, and why our benchmark answers got more accurate after we compressed context to 8% of its size.

The window is a constant. The channel isn't.

A channel is the basic unit of collaboration in Kylon, and every channel has an agent in it. People @-mention it to hand off work; it reads the discussion, calls tools, and reports back. After three months of this, a busy channel easily accumulates over a million tokens of messages and tool output, while the model can only see a fixed amount at once (300K tokens on our default model tier).

Think of the window as a desk of fixed size. The pile keeps growing; the desk doesn't. The question is never whether to tidy up. It's how to tidy up without losing anything. Because for a working agent, history isn't chat scrollback; it's working memory. A scenario from any week in Kylon: three weeks ago the team decided "cap exports at 50 rows per page." Today someone asks the agent to "generate the report using the pagination rule we agreed on." It has to remember the 50, and why it wasn't 100. Tidy the desk wrong, and questions like that get a confident, wrong answer.

Two standard fixes, two ways to forget

The industry has two default answers, and each throws away a different half of the problem:

• Drop the oldest messages (a sliding window). Trivial to build. But anything past the window edge simply stops existing. Last week's decision is gone by this week.
• Summarize everything into one blob (one-shot compaction). More dignified, but it fails twice. The swap happens at a single instant, the agent's memory is suddenly replaced wholesale, and its behavior drifts with it. Worse, it's irreversible: any detail the summarizer didn't keep is unrecoverable. And you can't know at compression time which details will matter three weeks later.

Fig. 1 — Three ways to handle a growing transcript. The first two lose information. The third changes how information is presented, not whether it exists.

So the requirements are two, and both are hard: the context must stay small (or it won't fit), and the details must stay recoverable (or the agent slowly loses its mind). These sound contradictory, until you separate compressing information from deleting it.

Fold history into a tree

Kylon's answer is a lossless context compression engine. The intuition is homely. It's how you'd tame an overflowing filing cabinet:

• Box up the oldest material. A small model reads the oldest stretch of conversation and writes a summary card. The originals go into the archive untouched; only the card stays on the desk.
• When boxes pile up, box the boxes. Once several adjacent cards accumulate, they're merged into a single higher-level card. The older the history, the deeper it's folded.
• Every card remembers what it covers. A card isn't a replacement for the originals. It's an index to them. You can always follow it back down to the exact words.

The result is a tree: verbatim originals at the bottom, progressively broader summaries above. Each turn, the model sees only the tree's top view, the current row of cards, plus the most recent messages kept raw (the newest ones are never folded). Rather than read more definitions, watch one grow:

A channel grows from zero to 36 messages (parameters shrunk for the demo: fold every 6 old messages into a card, never fold the last 4, roll up every 3 same-level cards; production values below). Two things to notice: folded originals only dim, they never disappear, and no matter how long the history gets, the strip at the top stays short.

The production rules fit in a sentence each. A summary card ingests up to 20K tokens of source and comes out around 1,200, a 16:1 fold. The most recent 32 messages are always kept verbatim. Four adjacent same-level cards roll up into one. Summarization prompts are depth-aware: bottom-level cards are told to preserve names, numbers, dates, file paths, and stated preferences; higher-level cards keep only decisions still in effect and their rationale. And one blunt safety rule guards the whole structure: if a summary isn't shorter than its source, it's rejected, every level of the tree is strictly smaller than the one below it.

What "lossless" actually means

Folding alone isn't enough. If the agent could only read cards, this would just be forgetting with extra steps. So every agent gets three recall actions: search the full archive (originals and cards), inspect a card's coverage, and unfold a card level by level, down to the verbatim messages. The click you just made in the demo, the agent does that for itself, whenever it needs to.

"Summaries above are compressed context. Maps to details, not the details themselves."

From the system prompt we inject whenever summaries are present

We back the sentence with a rule of conduct: for precision questions, exact commands, paths, timestamps, config values, root-cause chains, the agent must not answer from summaries alone. It retrieves the detail first, or says it isn't sure. "Lossless" isn't a storage slogan; it's a behavioral contract. Folded is not forgotten.

Threads fork. Memory has to fork with them.

Everything so far would work for a single conversation on a single machine, which is exactly what the paper assumes. Kylon broke that assumption in the first week. In the product, a thread is a first-class object: hover any message, say the one where the team was debating watermark options, and branch the discussion off. The agent follows you in. For that to work, the thread must start with the channel's memory up to the fork point (or it won't understand its own origin), then diverge freely (or the two discussions would contaminate each other).

With a per-conversation tree, there are only two options, both bad: copy the channel's cards into every new thread (write cost proportional to all of history, per fork), or share one mutable tree (the channel's next fold silently rewrites what the thread remembers). We wanted neither.

The fix is to separate the tree from the act of looking at it:

• Cards and originals belong to the whole family of conversations, and are immutable. Anything folded before the fork can be referenced by every branch, forever.
• Each conversation, channel or thread, owns only a catalog: which cards and which raw messages it sees, in what order. Folding and forking rewrite your own catalog, never the shared cards.

Fig. 2 — A fork in one picture: shared cards, private catalogs. After the fork, the channel rolls three cards into one; the thread's catalog doesn't move. Because cards are immutable, no branch can ever pull history out from under another.

Forking becomes cheap and exact: trim a snapshot of the parent's catalog at the fork point. Cards that fall entirely before the fork are referenced as-is. The one card that straddles the fork is expanded one level and backfilled with raw messages. No summary text is copied, and the cost is independent of how long the history is. It's copy-on-write, applied to memory.

Compaction runs in the background

The second production problem is timing. Folding calls a model and takes tens of seconds. It must never sit on the path where a user is waiting for a reply. From inside a channel, the only thing you should ever notice is that the agent stays fast and stays sharp, no matter how old the channel gets.

There's also a subtle measurement trap. Pressure on the window doesn't come from the transcript alone. A large share of every real prompt is structural: the system prompt, tool definitions, reasoning. Watch only the transcript and you'll hit a blind spot where the chat looks small but the model is already at the ceiling. So we weigh both, the transcript's own size and what the model actually carried last turn, and act on the larger. Two watermarks split the responsibility:

• Soft line (210K tokens): time to tidy. Schedules background folding between turns. Users never notice.
• Hard line (300K tokens): out of road. Rare, say a turn ingested a huge file. Folding runs immediately, in the foreground, with protective limits loosened. Structural rules (a rollup must merge at least two cards) never loosen.

Fig. 3 — Folding as housekeeping: each finished turn leaves a note, a scanner catches anything missed, and workers stay out of each other's way. Every dip on the left chart happened without anyone noticing.

One scheduling detail pays for itself: recently active channels are deliberately skipped. Their prompt prefix is probably still warm in the inference provider's cache, and folding right then would destroy and rebuild it at full price. Housekeeping that waits a few minutes is cheaper housekeeping.

Shared memory needs boundaries

Summary cards are inherited by forks, and a thread's audience isn't always the channel's audience. So "what is a summary allowed to remember" became a permissions question. The sharpest case: the agent sometimes quotes other channels or DMs it can read to inform its current answer. That material's visibility follows the agent, not the readers of this channel. Our rule is absolute: cross-channel quotations never enter summary text, and recall refuses to hand them to other branches. A summary that remembered too much would effectively declassify the information to every fork.

Benchmarks

We used a curated subset of LongMemEval (16-18 questions per configuration) to test whether folding + recall tools actually preserves accuracy.

Fig. 4 — Each point is one engine configuration. Every green point sits above and to the left of the blue baseline: with retrieval as a safety net, the engine can fold harder and answer better.

This is the moment "lossless" stops being a slogan and becomes a measurable property: accuracy is no longer bounded by summary density. Summaries tell the agent where to look; tools bring the detail back. Together, size and fidelity stop being a trade-off.

Methodology note: 16-18 questions per configuration, so read trends, not decimals. Folding and answering used a small, fast model. Production uses a stronger summarizer, so treat these as a floor. Grading combined string matching with a stronger LLM judge. "Folded size" is tokens reaching the model after folding divided by tokens of the original conversation.

What we borrowed, what we built

The folding core, the tree, the protected recent tail, the recall actions, follows Voltropy's Lossless Context Management paper, and we're glad it exists. Everything a team product needs around that core is what we ended up building:

What's next is already sketched: teaching the background worker to schedule around inference-cache lifetimes, tuning fold depth on longer horizons, and running the 500K-token benchmark tier end to end. The core judgment won't change, though. For an agent that lives with your team, forgetting shouldn't be a cost-saving side effect. It should be a view operation, one you can undo. Fold, don't forget.