Honcho, from an ML perspective

This note describes the technique, assumptions, and evaluation criteria for each Honcho component.

1. Defining Honcho

Honcho is memory infrastructure that gives AI agents persistent memory of users, agents, groups, and other entities across sessions.

Unlike approaches that rely on retrieval over text, Honcho models memory as reasoning over beliefs. It builds and continuously updates a natural-language representation of each entity, then reasons over that representation when queried.

Operationally, Honcho sits behind the agent as a dedicated memory layer. The agent sends interaction history to Honcho, and when it needs to act, asks Honcho for relevant context to include in its prompt.

2. The pipeline

Honcho separates memory formation from memory use.

Conversation turns enter an asynchronous write path (memory formation). The deriver (§4) extracts observations, the summarizer (§7) updates the session summary, and the dreamer (§5) revises the representation (§3) and peer card (§5). This path builds the per-peer representation over time, outside the agent’s synchronous query loop (memory use).

flowchart LR
    AGENT(["Agent"])
    MSG["Messages<br/>(conversation turns)"]
    DER["Deriver"]
    DREAM["Dreamer"]
    SUM["Summarizer"]
    REP[("Representation")]
    CARD[("Peer card")]
    SUMM[("Session summary")]

    AGENT -- posts conversation --> MSG
    MSG --> DER
    MSG --> SUM
    DER -- explicit --> REP
    DER -. triggers .-> DREAM
    REP -- reads --> DREAM
    DREAM -- deductive + inductive --> REP
    DREAM --> CARD
    SUM --> SUMM

On the read path (memory use), the agent asks Honcho a question. Honcho answers — either by producing an answer via the dialectic (§6) or with a context blob via get_context (§7) — using the representation constructed by the write path.

flowchart LR
    AGENT(["Agent"])
    MEM[("Memory")]

    AGENT -- question --> DIA["Dialectic (chat)"] --> ANS["answer"]
    AGENT -- request --> GC["get_context"] --> BLOB["context blob"]
    MEM -- reasons over --> DIA
    MEM -- assembles --> GC

3. The representation

Honcho stores memories primarily as natural-language propositions (“observations”), rather than as embeddings or entries in a formal symbolic store (embeddings are stored, but unlike in RAG, they are used only as an index for similarity retrieval over those observations). Each observation carries an epistemic level within a derivation hierarchy. Within that hierarchy, deductive beliefs are inferred from explicit facts, and inductive beliefs generalize over explicit facts and deductive beliefs.

explicit — a directly stated fact, extracted from a message.
deductive — a belief inferred from explicit facts (a logical consequence or a knowledge update), carrying its premises and links back to those facts.
inductive — a generalization over explicit facts and deductive beliefs (a pattern, trait, or preference), with a confidence grade.
contradiction — a flagged conflict between observations (not a level in the hierarchy).

Two principles govern the representation:

Provenance DAG: higher-level beliefs link to the observations that justify them, so a deduction is traceable to its premises and an answer can be grounded in evidence. The link requirement is enforced at write time — a deductive or inductive observation is rejected unless it cites its source observations. Consequently, the provenance path always exists and can be traversed.
Theory-of-mind scoping: memory is keyed by an (observer, observed) pair rather than globally. Under scoping, a peer’s self-model is (A, A), and A’s model of B is (A, B), filled only by what A was present to observe. Facts B revealed elsewhere do not enter A’s model of B. This boundary is enforced by keyed storage and ingest routing, not by model reasoning. Specifically, each (observer, observed) pair has its own store, populated only by messages from sessions in which the observer was present. The deriver, dreamer, and dialectic therefore operate only over the observations in that observer’s scoped store, without having to infer the boundary.

4. Deriver

The deriver performs open information extraction over noisy, multi-speaker dialogue. It maps context-dependent conversation spans to decontextualized, attributed, dated assertions about one target peer. The deriver only extracts explicit facts, deferring all meta belief formation to the dreamer (§5).

The deriver model (an LLM) receives a fixed prompt (the Deriver Prompt) that specifies the extraction task. The prompt directs the model to read a batch of conversation turns, use the non-target speakers as context, and emit atomic, self-contained facts about the target peer. The prompt defines explicitness, attribution, date normalization, and output format through instructions and worked examples. Constrained decoding enforces a structured schema in which each output contains a list of observation objects.

Extraction quality reduces to three criteria: i) the observation set must cover salient explicit facts from the source conversationl ii) each observation must be grounded in the cited source span and correctly attributed to the target peer; and iii) each observation must be decontextualized and minimal, expressing a single claim.

The deployed deriver runs as a single LLM call per batch of messages. Messages are batched by token count, with the immediately preceding turn from another speaker included for context.

Training

The deriver’s traning dataset is generated by converting a raw conversation corpus into grounded traces. First, the corpus is deduplicated, filtered to remove low-quality and role-play content, and decontaminated by dropping conversations that overlap with examples in other training corpora. The remaining conversations are then stratified by domain, length, and fact density. During training, conversations are sampled from those strata to preserve coverage across all three axes.

The deriver is distilled from a larger teacher model using this dataset. Each training example comprises the Deriver Prompt and a target: a serialized conversation with the subject peer marked, paired with a JSON list of observations. Each observation records a fact, the source message index, and a verbatim source span from that message.

For each conversation, the teacher model generates a set of target observations. A grounding check then keeps only proposed observations for which source span appears as a verbatim (after some basic normalization) substring of the cited message. Separate judge pairs then score the surviving observations for atomicity and redundancy. Each judge emits a continuous score from 0.0 to 1.0, which is binned into one of four ordinal classes before comparison. The judges “agree” when both assign an observation to the same bucket. Cohen’s kappa summarizes bucket-level judge agreement separately for atomicity and redundancy, correcting for chance co-classification. A run is rejected only if both the atomicity and redundancy kappas fall below threshold; that is, at least one agreement axis must pass. This allows item-level disagreement but rejects runs without reliable aggregate agreement on either axis.

The accepted, grounded traces become the SFT targets. Fine-tuning minimizes next-token cross-entropy on the target JSON, with prompt tokens excluded from the loss.

$L (θ) = - \sum_{t \in target} lo g p_{θ} (y_{t} ∣ y_{< t}, x)$

Here $x$ is the prompt plus the marked conversation, and the target tokens are the observation JSON.

Evaluation

The trained deriver generates traces — the source conversation and the deriver-generated observations for that conversation — from the benchmark environments (§8). The Deriver Benchmark Suite uses LLM judges to score the deriver from those traces along three axes: i)recall; ii) validity; and iii) well-formedness.

Coverage measures recall. A judge (LLM) reads the source conversation and extracts a gold set of facts, with each fact tagged by importance. The judge then evaluates each gold fact against the deriver’s observations, assigning one of three labels — covered, partially covered, or missing. Recall is the covered fraction of the gold set, $R = ∣ covered ∣/∣ G ∣$ ; a weighted variant scales each fact by importance, $R_{w} = (\sum_{g} w_{g} s (g)) / \sum_{g} w_{g}$ with $s (g) \in {1, \frac{1}{2}, 0}$ for covered, partial, and missing. Coverage also computes an F1, but with precision fixed at 1 (as in current implementation) it reduces to $F_{1} = 2 R / (1 + R)$ , a function of recall alone (i.e., scaled recall).

Importance	Judge rubric	Weight
Critical	Core identifying information (e.g., name, occupation, location, key relationships)	3.0
Important	Significant details that build a clear picture of the subject	2.0
Minor	Useful but nonessential details	1.0
Trivial	Marginal information	0.5

Note that because the gold set is synthesized by an LLM from the source conversation, Coverage measures recall against the judge model’s view of the salient facts — not from human-annotated ground truth.

Deriver Quality measures validity. For each deriver-generated observation, the judge sets boolean indicators for hallucination, over-inference, attribution failure, misattribution, misframing, meta-conversational content, and tautology. The judge separately evaluates the observation set for redundancy, topical fixation, and temporal confusion. These set-level checks are computed across observations and return the observations involved. An observation is invalid if any of the seven observation-level indicators is true or if the temporal-confusion check returns that observation (note that redundancy and topical fixation are tallied but do not affect the score). Deriver Quality reports the fraction of observations that remain valid, $score = ∣ {o : valid (o)} ∣/∣ O ∣$ with $valid (o) = \neg (⋁_{k} f_{k} (o) \lor tc (o))$ over the seven observation-level indicators $f_{k}$ and the temporal-confusion check $tc$ . Importantly, Deriver Quality evaluates only the observations produced by the deriver (i.e., the deriver is not penalized for missing facts as this is scored by Coverage).
Molecular measures well-formedness. The judge scores each observation on decontextuality and minimality. Decontextuality measures whether the observation can be interpreted on its own. Minimality measures whether the observation contains a single claim. The per-observation score is the geometric mean of the two, $mol (o) = decon (o) \cdot min (o)$ . Molecular isolates shape from truth, so a well-formed false observation can score high.

5. Dreamer

The dreamer operates offline on one representation at a time — i.e., a single (observer, observed) pair (§3) — after the deriver has written into that representation. Specifically, when the deriver writes a new explicit observation, the dreamer checks whether enough new observations have accumulated in the representation and whether a minimum interval has passed since the last dream. If both conditions hold, the dreamer schedules a dream for that representation.

When dreaming, the dreamer invokes two specialists in sequence over the representation. The deduction specialist reads the representation’s explicit observations, updates the belief set by applying direct implications, supersessions, and contradictions, writes deductive beliefs back into the same representation, and prunes stale deductive beliefs (all via tool calls). The induction specialist then reads the updated belief set, generalizes over the explicit and deductive observations, and writes inductive beliefs back into the same representation (via tool calls). The induction specialist is purely additive and cannot remove beliefs.

Each specialist is an LLM that produces representation edits through a two-phase loop. In the discovery phase, the specialist surveys the representation. It pulls recent observations, uses those observations to retrieve related observations by embedding similarity, and reads the source messages that support the retrieved observations. Given that evidence, the specialist enters the action phase, during which the specialist writes new observations and, in the case of the deduction specialist, deletes superseded observations. A specialist repeats discovery and action phases until the specialist can no longer make supported edits.

The specialists

The deduction specialist performs belief revision and truth maintenance. Its prompt directs it to make three kinds of edits. First, to make knowledge updates — wherein the same fact takes a new value — by writing the updated belief and deleting the outdated observation. Second, to draw logical implications, such as skills or status implied by a stated role. Finally, to identify contradictory statements. Every deductive observation must cite the source observations that support the deduction. Without such citations, the write is rejected. The deduction specialist also maintains the peer card, a compact identity store that holds only stable markers, including canonical identity, durable attributes, relationships, and explicitly stated standing instructions. Behavior, transient state, and inferred preferences remain in observations (§3).

The induction specialist performs generalization. Its prompt directs the specialist to read across explicit and deductive observations and write patterns — including preferences, behaviors, personality traits, and temporal trends — to the observations. Each inductive observation cites its supporting observations and records both a pattern type and a confidence grade. The confidence grade is scaled by the number of supporting sources. A pattern must have at least two supporting observations before the induction specialist writes the pattern.

Training

The dreamer is not trained — its behavior is entirely prompt driven.

Evaluation

The dreamer has no dedicated evaluation. Its behavior is measured only indirectly through end-to-end dialectic tests (§6).

6. Dialectic (`chat`)

The dialectic answers natural-language questions about a peer. At query time, a calling agent sends Honcho a question such as “What is this user’s preferred way of doing X?” Honcho returns a synthesized answer grounded in that peer’s stored representation, which the calling agent injects the answer into its own prompt.

The dialectic is an LLM agent that operates over the peer representation (§3). It draws on the representation, the underlying evidence store, and the peer card as needed; decides what to retrieve; determines when the evidence is sufficient; and synthesizes an answer.

This evidence-selection begins with a prefetched evidence set. When observations are added to the representation (by either the deriver or dreamer), Honcho computes and stores observation embeddings; when the dialectic receives a question, it embeds the question and runs a vector-similarity search over those observation embeddings.

After prefetching, the dialectic follows a Retrieval Policy encoded in a natural-language prompt. The policy gives the dialectic criteria for choosing tools, continuing or stopping retrieval, and deciding how cautiously to frame the final answer.

The dialectic can call tools that search observations semantically, grep messages for exact text, filter messages by date, retrieve message context around an observation, and traverse the provenance graph connecting observations. When the dialectic finds a deductive or inductive observation that answers the querying agent’s question, it follows those links back toward the explicit premises, verifies that they support the derived belief, and cites the concrete evidence rather than the synthesis alone. If the premises are weak, stale, or indirect, the answer qualifies the claim rather than presenting it as settled.

The dialectic also handles updates and conflicts at query time. When it finds a value that may have changed, it searches for update language and returns the most recent supported value, superseding older evidence. When it finds genuinely conflicting statements, it presents the conflict in its response rather than choosing a side. When the evidence is too thin, it abstains.

Reasoning depth is configurable. Each depth level maps to a model choice and a tool-iteration budget, trading latency and cost for more exhaustive retrieval and verification.

Training

The dialectic is not trained — its behavior is entirely prompt driven.

Evaluation

Evaluation follows the same query-time path. Honcho ingests a benchmark-supplied conversation history (§8) and a question about one peer. The deriver and dreamer run, creating a representation. The dialectic then reads that peer representation, retrieves supporting observations and source messages, follows provenance for derived beliefs, and returns an answer. The grader (defined in §8) scores that answer against the benchmark evidence. Note that the evaluation thus measures Honcho end to end — it does not isolate the effect of the dialectic.

To further contextualize Honcho’s performance, each benchmark is also run without Honcho by placing the entire conversation in a single LLM prompt. The Honcho-minus-baseline delta measures the contribution of memory over raw context.

7. `get_context`

get_context is the system’s context-assembly interface. It prepares the material for the querying agent’s own LLM call, but does not reason over that material itself.

The returned object is an assembled context blob. It includes recent messages, the session summary, and, optionally, a query-relevant slice of the peer’s representation (generated via semantic retrieval) together with the peer card. These pieces are packed to a token budget and returned as structured fields.

The packing gives priority to the representation slice and peer card. When they are included, their full token count is subtracted from the overall budget before allocating space to anything else; there is no fixed percentage cap on these tokens. Whatever remains is divided between the session summary and recent messages, with the summary capped at 40% of that remainder.

$B^{'} = B - tok (R) - tok (P), tok (S) \leq 0.4 B^{'}, tok (M) \leq B^{'} - tok (S)$

Here $B$ is the total budget, $R$ the representation slice, $P$ the peer card, $S$ the chosen summary, and $M$ the recent-message window.

The session summary is defined by the summarizer. On a periodic message cadence, the summarizer updates both a short and a long summary of the session. The long summary is used if it fits (and is strictly longer than the short summary), otherwise the short summary if that fits, otherwise no summary. The recent-message window is then packed into the remaining token budget.

Importantly, unlike the dialectic, get_context does not reason over the representation. It is nearly a fully deterministic, token-budgeted selection procedure. There is just one generative sub-component — summarization. Both the long and short summaries are updated by an LLM prompted to combine the previous summary together with the new messages, then recompressing the result under a (soft) word limit. This summarization step uses a configured, off-the-shelf model. The summarizer is not trained.

Evaluation

get_context has no dedicated eval. Instead, the dialectic benchmarks can be rerun using get_context rather than through the dialectic. Summarization, the generative step, is not scored separately. Its quality appears only indirectly, through its effect on those benchmark scores.

8. Benchmarks

Benchmark	Capability	Structure	Grader
LongMem	Six question types: single-session recall (of a user fact and of an assistant fact, scored separately), multi-session recall, temporal reasoning, knowledge update, and single-session preference; plus unanswerable questions for abstention	Multi-session haystack with distractor sessions; the relevant fact may sit anywhere in history	Binary LLM judge, per-type pass rate
LoCoMo	Very-long-term conversational QA	Multi-session, two speakers, dated; adversarial questions excluded by default	Binary LLM judge (sufficient/insufficient against resolved evidence)
BEAM	Ten memory abilities, including contradiction resolution, event ordering, knowledge update, instruction following	Conversation sized in fixed buckets; full ingest	Kendall tau-b for event ordering; nugget-based LLM judging otherwise

Plastic Labs Internal Wiki

Explorer

Recently Updated

Company

Operating Model — Decisions Log

Operating Model

Security & Compliance

ideal-reasoned-representations

System Overview

Honcho, from an ML perspective

1. Defining Honcho

2. The pipeline

3. The representation

4. Deriver

Training

Evaluation

5. Dreamer

The specialists

Training

Evaluation

6. Dialectic (`chat`)

Training

Evaluation

7. `get_context`

Evaluation

8. Benchmarks

Graph View

Table of Contents

Backlinks

Plastic Labs Internal Wiki

Explorer

Recently Updated

Company

Operating Model — Decisions Log

Operating Model

Security & Compliance

ideal-reasoned-representations

System Overview

Honcho, from an ML perspective

1. Defining Honcho

2. The pipeline

3. The representation

4. Deriver

Training

Evaluation

5. Dreamer

The specialists

Training

Evaluation

6. Dialectic (chat)

Training

Evaluation

7. get_context

Evaluation

8. Benchmarks

Graph View

Table of Contents

Backlinks

6. Dialectic (`chat`)

7. `get_context`