Plastic Labs Internal Wiki

Recently Updated

rfc-composable-peers

8 min read

RFC: Composable Peers v2 — Universal Identity Containers for Honcho

Status: Draft (v2) Owner: vineeth Last updated: 2026-03-25

Changelog from v1: This v2 update incorporates decisions from the full spec-writing process. Key changes:

  • Peer files (Section 4.3) extracted to standalone file-system-primitives.md spec with versioning, auto-embedding, and storage backend abstraction
  • Materialized dreams (Section 4.5) extracted to standalone dreaming-enhancements.md spec with TaggingSpecialist, dream_definitions table, and consolidation improvements
  • Custom instructions (Section 4.2) confirmed as in-progress work — field exists in config schema but not yet injected into prompts
  • Non-goals updated: multi-modal support now has its own spec (multi-modal.md), so binary files are handled there, not here
  • Conclusion tagging (public metadata on documents) now has its own spec (conclusion-tagging.md), which this RFC depends on
  • Reasoning traces now have their own spec (reasoning-traces.md), enabling the provenance chain needed for invalidation (Section 4.4)
  • Implementation phases updated to reflect dependency ordering across all specs

Original preserved at: rfc-composable-peers.v1.md

1. Problem Statement

(Unchanged from v1 — the core problems remain.)

Honcho’s peer model is intended to represent arbitrary concepts — people, agents, groups, organizations, projects — but in practice peers are treated as individual interlocutors in conversations. The system’s rigidity manifests in:

  1. Peers are not composable. No way to express “engineering-team contains alice, bob, charlie” or segment a peer’s identity.
  2. Data flows only through conversations. No way to feed arbitrary documents or external data without synthetic sessions.
  3. No per-peer extraction guidance. The deriver treats every peer identically. (Being addressed by custom instructions — in progress.)
  4. The reasoning chain is immutable. No rename, merge, split, or correction operations.
  5. No developer-extensible reasoning. (Now addressed by dreaming-enhancements.md.)
  6. Files aren’t native. (Now addressed by file-system-primitives.md.)

Competitive Context

(Unchanged from v1 — Supermemory analysis still applies.)

2. Goals

  1. Sub-peer composition — any peer can have sub-peers, any peer can belong to multiple parents, knowledge aggregates upward.
  2. Custom instructions — per-peer, per-session, per-message extraction context. (In progress separately.)
  3. Peer files — peer-scoped versioned file storage with semantic search. (Detailed in file-system-primitives.md.)
  4. Restructuring operations — rename, merge, split with managed invalidation.
  5. Materialized dreams — developer-defined recurring investigations. (Detailed in dreaming-enhancements.md.)
  6. Session-level filtering — fine-grained isolation in dialectic and conclusion queries.
  7. Dialectic enhancement — new tools for sub-peer navigation, file search, dream outputs. (Detailed in dialectic-enhancements.md.)

3. Non-Goals

  1. Storing binary files. → Binary support now handled by multi-modal.md spec (images, PDFs via external storage).
  2. Turning Honcho into a general-purpose vector database.
  3. Breaking backward compatibility.
  4. Introducing a new entity type. Sub-peers are just peers.

4. Design

4.1 Sub-peers: Hierarchical Peer Composition

(Core design unchanged from v1.)

A peer can have sub-peers, and a peer can belong to multiple parent peers. Sub-peers are regular peers with a membership relationship.

Knowledge aggregation is directional:

  • Query a parent peer → sees parent’s own knowledge + all sub-peers’ knowledge
  • Query a sub-peer → sees only that sub-peer’s knowledge

Schema:

CREATE TABLE peer_memberships (
    id TEXT PRIMARY KEY,
    workspace_name TEXT NOT NULL,
    parent_peer TEXT NOT NULL,
    child_peer TEXT NOT NULL,
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMPTZ DEFAULT now(),
    UNIQUE (workspace_name, parent_peer, child_peer),
    FOREIGN KEY (parent_peer, workspace_name) REFERENCES peers(name, workspace_name),
    FOREIGN KEY (child_peer, workspace_name) REFERENCES peers(name, workspace_name)
);

API surface:

POST   /v3/workspaces/{id}/peers/{parent}/members           -- add sub-peer
GET    /v3/workspaces/{id}/peers/{parent}/members           -- list sub-peers
DELETE /v3/workspaces/{id}/peers/{parent}/members/{child}   -- remove sub-peer
GET    /v3/workspaces/{id}/peers/{peer_id}/parents          -- list parent peers

Aggregation behavior:

  • Default depth: 1 (self + immediate children)
  • Configurable via depth parameter: 0 = self only, 1 = default, 2 = two levels, all = full tree
  • Cycle detection at membership creation time

Many-to-many membership: A peer can belong to multiple parents. Knowledge aggregates upward to all parents.

Interaction with observer/observed model: Collections store conclusions at the individual peer level. Sub-peer aggregation happens at query time, not storage time.

4.2 Custom Instructions

(In progress separately — field exists in ReasoningConfiguration.custom_instructions but is not yet injected into prompts.)

Peers, sessions, and messages each gain a freeform custom_instructions field that the deriver incorporates into its extraction prompt:

  • Peer-level: what kind of entity, what to focus on
  • Session-level: what this session represents
  • Message-level: what this specific content is

4.3 Peer Files

→ See file-system-primitives.md for full design.

Summary of what changed from v1:

  • Versioning added: Each file write creates a new version (PostgreSQL file_versions table). Full edit history preserved.
  • Auto-embedding: Files are automatically chunked and embedded for semantic search at the peer level.
  • Storage backend abstraction: Configurable backend (PostgreSQL for text, S3/GCS/local for large content).
  • Agent tools: search_files, read_file, grep_files tools for dialectic and dreamer.

4.4 Reasoning Chain Invalidation

(Updated from v1 to leverage reasoning traces from reasoning-traces.md.)

Approach: Staleness propagation with dreamer reconciliation.

  1. Staleness flag. Documents (conclusions) gain stale boolean and stale_reason text in internal_metadata.
  2. Source tracking. Documents already have source_ids. When a source is marked stale, downstream conclusions referencing it are also marked stale.
  3. Dreamer reconciliation. The dreamer gains a reconcile_stale task type (see dreaming-enhancements.md consolidation improvements).
  4. Reasoning trace linkage. Each conclusion links to its reasoning trace (see reasoning-traces.md). When reconciling, the dreamer can inspect the original reasoning to decide whether the conclusion is still valid.

Restructuring operations:

OperationMechanism
Delete messageTombstone message; mark directly-derived conclusions stale; dreamer reconciles
Rename peerCreate new peer; migrate memberships, files, conclusions; tombstone old peer
Merge peersCreate target peer; migrate from sources; dreamer resolves duplicates; tombstone sources
Split peerCreate sub-peers; move conclusions by criteria; mark affected conclusions stale
Correct conclusionUpdate content; mark downstream conclusions stale; dreamer reconciles

API surface:

POST /v3/workspaces/{id}/peers/{peer_id}/rename    -- {new_name: str}
POST /v3/workspaces/{id}/peers/merge               -- {sources: [peer_ids], target_name: str}
POST /v3/workspaces/{id}/peers/{peer_id}/split      -- {criteria: {...}, sub_peer_names: [str]}

4.5 Materialized Dreams

→ See dreaming-enhancements.md for full design.

Summary of what changed from v1:

  • dream_definitions table with full CRUD API.
  • Three trigger types: cron (scheduled), on_change (debounced), on_demand (explicit API call).
  • Three output types: conclusions (stored as tagged observations), metadata (stored in collection internal_metadata), prose (written to peer files when file system is available).
  • TaggingSpecialist: New specialist that auto-generates tags on observations based on patterns.
  • Consolidation improvements: Near-duplicate detection, staleness scoring, capacity pressure.

4.6 Session-Level Filtering

(Unchanged from v1.)

Dialectic and conclusion query endpoints gain session-level filtering:

honcho.peers.get_representation(
    peer_id="alice",
    session_ids=["session-123", "session-456"]
)
 
honcho.peers.get_representation(
    peer_id="alice",
    exclude_session_ids=["private-session-789"]
)

Composes with sub-peers and metadata filters.

4.7 Dialectic Enhancement

→ See dialectic-enhancements.md for structured outputs, evidence, and completions endpoint.

Additional tools for composable peers:

  • list_sub_peers(peer_id) — discover sub-peers
  • list_parent_peers(peer_id) — discover parents
  • search_files(peer_id, query, top_k) — semantic search across peer’s files (from file-system-primitives)
  • read_file(peer_id, path) — read a peer file
  • get_materialized_dream(peer_id, dream_name) — read a dream output

Existing tools (search_memory, get_representation) automatically include sub-peer knowledge when called on a parent peer.

5. Mapping: Endearing’s Supermemory Architecture

(Unchanged from v1.)

Supermemory PatternComposable Peers Equivalent
user_{id} containerParent peer — query for full knowledge
groupchat_{chatId} containerSub-peer — query for scoped knowledge
user_{id}-chat_{chatId}Sub-peer or session filter
Entity context per containerCustom instructions per peer
client.profile()Representation on parent peer + materialized dream
Fan-out searchQuery parent peer (aggregates automatically)
Container isolationQuery specific sub-peer
Container mergingPeer merge operation with dreamer reconciliation

6. Schema Changes Summary

New tables (this RFC only — files and dreams have their own specs):

CREATE TABLE peer_memberships (
    id TEXT PRIMARY KEY,
    workspace_name TEXT NOT NULL,
    parent_peer TEXT NOT NULL,
    child_peer TEXT NOT NULL,
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMPTZ DEFAULT now(),
    UNIQUE (workspace_name, parent_peer, child_peer),
    FOREIGN KEY (parent_peer, workspace_name) REFERENCES peers(name, workspace_name),
    FOREIGN KEY (child_peer, workspace_name) REFERENCES peers(name, workspace_name)
);

Modified tables:

  • documents: Add stale and stale_reason to internal_metadata (no schema migration — JSONB field)

Dependencies on other specs:

  • conclusion-tagging.md — public metadata column on documents (must land first)
  • file-system-primitives.md — peer_files and file_versions tables
  • dreaming-enhancements.md — dream_definitions table
  • reasoning-traces.md — reasoning_traces table (for invalidation provenance)

7. Implementation Phases

Phase 1: Sub-peers (depends on: nothing)

  • Implement peer_memberships table and CRUD endpoints
  • Aggregation logic in representation, conclusion queries, and search
  • Cycle detection at insertion time
  • Configurable aggregation depth
  • Validates: does sub-peer composition cover target modeling scenarios?

Phase 2: Session-Level Filtering (depends on: nothing)

  • Add session_ids and exclude_session_ids params to dialectic and conclusion queries
  • Filter on documents.session_name
  • Validates: does session filtering provide useful isolation?

Phase 3: Invalidation & Restructuring (depends on: conclusion-tagging, reasoning-traces)

  • Staleness propagation via source_ids DAG
  • Dreamer reconcile_stale task (from dreaming-enhancements)
  • Peer rename, merge, split API endpoints
  • Validates: can the system recover coherent state after restructuring?

Phase 4: Dialectic Integration (depends on: phases 1-3, file-system-primitives, dreaming-enhancements)

  • New tools: list_sub_peers, list_parent_peers, search_files, read_file, get_materialized_dream
  • Aggregation in existing tools when called on parent peers
  • Validates: does the dialectic effectively use hierarchical knowledge?

8. Open Questions

  1. Aggregation depth default. Should default be 1 (immediate children) or 0 (self only, opt-in)? Recommend: 1, with clear documentation.

  2. Collection model interaction. When querying a parent peer’s representation, should we fan-out to all sub-peer collections? Recommend: query-time fan-out (simpler, no aggregate collections to maintain).

  3. Staleness during active ingestion. The deriver should skip stale conclusions as premises. The dreamer should prioritize reconciliation over new dream tasks.

  4. Multi-parent knowledge exposure. A peer in multiple parents exposes its knowledge to all parents. Document this clearly — it’s by design but could surprise developers.

  5. Restructuring atomicity. Rename/merge/split operations should be atomic or use a state machine with rollback. Recommend: queue-based with progress tracking (similar to dream task pattern).

9. Success Criteria

  1. Model people, teams, brands, codebases, documents, and multi-agent systems using composable peers with sub-peers.
  2. Parent peer aggregates sub-peer knowledge. Sub-peer provides isolation. Both without special configuration.
  3. Peer rename, merge, split operations recover coherent state via dreamer reconciliation.
  4. All changes are additive — existing API consumers experience no breaking changes.
  5. SDK v2.1 with sub-peer management methods.

Appendix A: Modeling Scenarios

(Unchanged from v1 — all scenarios still apply. See rfc-composable-peers.v1.md Appendix A for full scenarios covering: People, Teams, Brands, Codebases, Documents, Autonomous Agents, Multi-Agent Orchestration.)

Graph View

Backlinks