ai-sdk-spec

@honcho-ai/tools — Integration Redesign Plan

Problem Statement

The current integration treats everything as a single peerId — both user and assistant messages get the same peer attribution. This doesn’t leverage Honcho’s multi-peer observation model and misses the key differentiator over other Vercel AI SDK memory providers (Mem0, Supermemory, Hindsight, Letta).

Additionally, the integration reimplements context formatting (defaultFormatContext()) instead of using Honcho’s native SessionContext.toOpenAI() / toAnthropic() converters, which already handle multi-peer → user/assistant role mapping.

Research Summary

How Other Vercel AI SDK Memory Providers Handle Identity

Provider	Identity Param	User vs Assistant Distinction	Session Concept
Letta	agent.id (1 agent = 1 user)	No — assistant handled server-side	Implicit (agent state)
Mem0	user_id + agent_id + run_id	Yes — roles preserved, multi-dimensional	run_id (manual)
Supermemory	containerTag (e.g. user ID)	No — flat text blobs	conversationId (middleware only)
Hindsight	bankId (typically user ID)	No — flat content strings	None

Key findings:

• All providers bind identity at configuration time, never per-message in tool calls
• Only Mem0 distinguishes user vs assistant as separate entities
• Session tracking is generally weak or absent across all providers
• Two integration shapes: provider wrappers (Mem0, Letta) and tools (Hindsight, Supermemory)

How the AI SDK Handles Sessions

The AI SDK is stateless — every generateText/streamText call takes explicit messages or prompt. There’s no built-in session object. Developers manage conversation state themselves. This works in Honcho’s favor because Honcho can be the session/state layer.

Honcho’s .context() Method

Two levels:

• Session.context() — returns SessionContext with messages, summary, peerRepresentation, peerCard
• Peer.context() — returns PeerContext with just representation + card

SessionContext.toOpenAI(assistantPeer) does smart role mapping:

• Messages from assistantPeer → role: "assistant"
• Messages from any other peer → role: "user", name: peerId
• Representation/card/summary prepended as system messages

SessionContext.toAnthropic(assistantPeer):

• Same mapping but user messages prefixed with "${peerId}: ${content}"
• Context uses role: "user" instead of role: "system" (Anthropic handles system separately)

The current integration does NOT use these converters — it reimplements with defaultFormatContext().

AI SDK Middleware System

Three hooks:

• transformParams — modify params before LLM call (inject context)
• wrapGenerate — wrap non-streaming generation (persist messages after)
• wrapStream — wrap streaming generation (collect chunks, persist after)

Middlewares compose: [first, second] applies as first(second(model)). Custom data passed via providerOptions: { yourName: { ... } }.

Honcho’s Differentiators

Feature	Mem0	Supermemory	Hindsight	Honcho
Separate user/assistant identity	Via role + agent_id	No	No	Peers with observation config
Theory-of-mind	No	No	No	Cross-peer representation
Dialectic reasoning	No	No	Partial (reflect)	honcho_chat
Async consolidation	No	No	No	Dreaming
Session-scoped context	Weak (run_id)	Weak (conversationId)	No	First-class sessions
Native LLM format conversion	No	No	No	toOpenAI() / toAnthropic()

---

Proposed Integration Patterns

Pattern A: “Honcho as Session Manager” (Opinionated — recommended for most apps)

Honcho owns the conversation state. .context() is the source of truth for history. New messages get persisted automatically, and the next call gets updated history.

import { createHoncho } from "@honcho-ai/tools/ai-sdk";
 
// 1. Create provider (one per app)
const honcho = createHoncho({
  workspaceId: "my-app",
  apiKey: process.env.HONCHO_API_KEY,
});
 
// 2. Get a session handle (lightweight config, no API call)
const session = honcho.session("conv-123", {
  user: "user-456", // → peer, observeMe: true
  assistant: "my-assistant", // → peer, observeMe: false
});
 
// 3. Generate — middleware handles context fetch + message persistence
const result = await generateText({
  model: baseModel,
  tools: session.tools(), // honcho_chat, honcho_search, etc.
  middleware: session.middleware(), // auto context + persistence
  prompt: "What should I focus on today?",
});

How middleware works internally:

• transformParams: calls Session.context() → toOpenAI(assistant) to get properly formatted history with representation/summary, injects into message array
• wrapGenerate/wrapStream: persists user message under user peer, assistant response under assistant peer — correct two-peer attribution

Dynamic session management in request handlers:

app.post("/chat", async (req) => {
  const session = honcho.session(req.body.sessionId, {
    user: req.auth.userId,
    assistant: "my-bot",
  });
  // session is lightweight — just config, no lifecycle management needed
});

Pattern B: “Multi-Peer Perspectives” (For multi-agent / group chat)

For apps with more than 2 peers — group chats, multi-agent orchestration, agents observing each other.

const honcho = createHoncho({
  workspaceId: "my-app",
  apiKey: process.env.HONCHO_API_KEY,
});
 
// Define all participants
const session = honcho.session("group-chat-789", {
  peers: [
    { id: "user-alice", observeMe: true, observeOthers: false },
    { id: "user-bob", observeMe: true, observeOthers: false },
    { id: "agent-coordinator", observeMe: false, observeOthers: true },
    { id: "agent-specialist", observeMe: false, observeOthers: true },
  ],
});
 
// Each agent gets middleware scoped to its perspective
// agent-coordinator sees its theory-of-mind of alice, bob, and specialist
const coordinatorModel = wrapLanguageModel({
  model: baseModel,
  middleware: session.middlewareFor("agent-coordinator"),
});
 
const specialistModel = wrapLanguageModel({
  model: baseModel,
  middleware: session.middlewareFor("agent-specialist"),
});
 
// Context is fetched from coordinator's POV, response persisted as coordinator's message
const result = await generateText({
  model: coordinatorModel,
  tools: session.toolsFor("agent-coordinator"),
  prompt: "Alice asked: can you help with X?",
});
 
// Cross-peer queries
const aliceProfile = await session.ask(
  "agent-coordinator", // observer
  "user-alice", // target
  "What are Alice's goals?",
);

How middlewareFor(peerId) works:

• transformParams: calls .context({ peerPerspective: peerId, peerTarget: ... }) to get that agent’s theory-of-mind, then toOpenAI(peerId) maps the calling agent to assistant role, everyone else to user role
• wrapGenerate/wrapStream: persists response attributed to the specific agent peer

Pattern C: “Bring Your Own Loop” (Maximum flexibility)

No middleware — just utilities. For custom pipelines, evals, batch processing, non-standard flows.

const honcho = createHoncho({
  workspaceId: "my-app",
  apiKey: process.env.HONCHO_API_KEY,
});
 
// Fetch context directly — you decide when and how
const ctx = await honcho.context("session-123", {
  assistant: "my-bot",
  peerTarget: "user-456",
  tokens: 4096,
  summary: true,
});
 
// Use Honcho's native converters
const history = ctx.toOpenAI("my-bot");
// or: ctx.toAnthropic("my-bot")
 
// Build your own message array
const result = await generateText({
  model: baseModel,
  messages: [...history, { role: "user", content: "New question" }],
  tools: honcho.tools({ peerId: "user-456" }),
});
 
// Persist manually
await honcho.addMessages("session-123", [
  { peerId: "user-456", content: "New question" },
  { peerId: "my-bot", content: result.text },
]);
 
// Trigger dreaming manually
await honcho.dream("session-123", {
  observer: "my-bot",
  observed: "user-456",
});

---

Pattern Comparison

Concern	Pattern A (Session Manager)	Pattern B (Multi-Peer)	Pattern C (BYOL)
Setup complexity	Low	Medium	High
Peers	2 (user + assistant)	N (any number)	N (manual)
Context injection	Automatic via middleware	Automatic, per-perspective	Manual
Message persistence	Automatic, correct peer attribution	Automatic, per-agent	Manual
Uses toOpenAI/toAnthropic	Yes (in middleware)	Yes (per perspective)	Yes (direct call)
Session management	Dynamic — .session(id, peers) per request	Dynamic — .session(id, {peers})	Manual
Best for	Standard chatbot/assistant apps	Multi-agent, group chat	Custom pipelines, evals

---

Key Design Decisions

1. Use Honcho’s native toOpenAI()/toAnthropic() instead of reimplementing context formatting. Gets proper multi-peer → role mapping for free.

2. Always create separate peers for user and assistant. Pattern A’s { user, assistant } config creates two peers with correct observation settings (observeMe: true for user, observeMe: false for assistant).

3. Session handles are lightweight config objects, not API connections. .session() captures IDs and peer configs. No object-per-session lifecycle management needed.

4. Two-peer is the common case, multi-peer is the escape hatch. Pattern A covers 90% of apps. Pattern B uses the same underlying machinery when you need it.

5. Middleware is opt-in, not the only path. Pattern C ensures Honcho stays flexible for non-standard use cases while still providing converters as the canonical bridge from multi-peer to LLM format.

6. Message persistence uses correct peer attribution. User messages → user peer, assistant messages → assistant peer. No more single-peerId for everything.

---

Issues with Current Implementation

1. Single peerId for both roles (context.ts:164-169) — both user and assistant messages get same peer_id. Should route by role to separate peers.

2. Reimplements context formatting (defaultFormatContext()) instead of using Honcho’s toOpenAI()/toAnthropic() converters.

3. Only persists last user message (middleware.ts:105-107) — in multi-step tool loops, intermediate messages may be lost.

4. Fire-and-forget persistence (middleware.ts:120) — silently swallows errors. No signal when messages are lost.

5. v2→v3 URL rewriting (context.ts:14-18) — temporary bridge with naive string replace that should be tracked for removal.

6. No provider detection — middleware always injects XML-formatted context into system prompt. Should detect provider and use toOpenAI() vs toAnthropic() accordingly.

---

Validation Test Plan

Category 1: Context Injection

• Context injected into empty system prompt
• Context appended to existing system prompt
• Context injection skipped when peerId missing
• Context injection skipped when injectContext=false
• Session context vs peer-only context paths
• targetPeerId enables cross-peer perspective
• System message with content parts array (not just string)

Category 2: Message Persistence

• User + assistant messages persisted with correct peer attribution
• Persistence skipped when persistMessages=false
• Persistence skipped when sessionId missing
• Stream persistence collects full text
• Persistence failure doesn’t block response
• Multi-turn: all new messages persisted (not just last)

Category 3: Peer/Session Identity

• Provider defaults used when no call options
• Call options override provider defaults
• Missing peerId returns empty context gracefully
• Two-peer setup creates correct observation configs
• Multi-peer setup with N peers

Category 4: Tools

• All tools available and correctly typed
• Tools scoped to correct peer
• Tools use session context when available

Category 5: E2E Scenarios

• Full conversation flow: context fetch → generation → persistence → next call sees persisted messages
• Multi-agent: two agents with different perspectives on same session
• Round-trip: persist then retrieve via .context() → toOpenAI()
• Dreaming cycle: add messages → dream → reflect → verify consolidated insights

Category 6: Native Converter Integration

• toOpenAI() produces correct role mapping for 2-peer session
• toAnthropic() produces correct format with peer prefixes
• Multi-peer toOpenAI() maps non-assistant peers to user role with name
• Token budget respected in context fetch