Plastic Labs Internal Wiki

Recently Updated

dialectic-enhancements

34 min read

Dialectic Enhancements: Structured Outputs, Evidence/Citations, and OpenAI-Compatible Endpoint

Status: Draft Owner: vineeth Last Updated: 2026-03-25 Target Release: v3.x (additive, non-breaking)

1. Problem Statement

The Dialectic API (POST /v3/workspaces/{id}/peers/{peer_id}/chat) is Honcho’s primary recall interface. It takes a natural language query, runs an agentic tool loop to gather context from memory, and returns a synthesized answer. Three gaps limit its usefulness:

1.1 No Structured Output

DialecticResponse returns content: str | None — plain text only. Applications that need machine-readable output (e.g., a recommendation engine expecting { "items": [...], "reasoning": "..." }) must parse the free-form string themselves. This is fragile, model-dependent, and error-prone.

The LLM layer already supports structured output: honcho_llm_call() accepts a response_model: type[BaseModel] parameter that instructs the provider to return JSON conforming to a Pydantic schema. This capability is unused by the Dialectic because there is no way for the caller to pass a schema through the API.

1.2 No Evidence or Citations

When the Dialectic answers “The user prefers dark roast coffee,” there is no way to verify this. The caller receives a string with no reference to which conclusions (observations) were accessed, which messages were searched, or what reasoning chain the agent followed. This makes the Dialectic a black box.

The evidence already exists internally: tool_calls_made on HonchoLLMCallResponse / StreamingResponseWithMetadata captures every tool call with its name, input, and result. The prefetch step also records which conclusion IDs were retrieved. This data is logged to telemetry and then discarded.

1.3 No OpenAI-Compatible Interface

Many AI application frameworks (LangChain, LlamaIndex, Vercel AI SDK, etc.) speak the OpenAI Chat Completions protocol. To use the Dialectic today, developers must write custom HTTP integration code. An OpenAI-compatible endpoint would allow Honcho to be used as a drop-in replacement for openai.ChatCompletion.create(), massively lowering the integration barrier.

2. Goals / Non-Goals

Goals

  • G1 (Structured Output): Accept a JSON Schema in DialecticOptions, run the normal agentic tool loop, and format the final synthesis call’s output to conform to the schema. Return the structured object in DialecticResponse.content.
  • G2 (Evidence): Add an evidence field to DialecticResponse containing the conclusion IDs, message IDs, and observation objects the agent accessed during the tool loop. Collect this from tool call tracking, not by asking the model.
  • G3 (OpenAI-Compatible Endpoint): Expose POST /v1/chat/completions that accepts OpenAI-format requests, maps them through the Dialectic, and returns OpenAI-format responses with SSE streaming support.
  • G4 (SDK Integration): Python SDK accepts a Pydantic model (auto-converted to JSON Schema). TypeScript SDK accepts a Zod schema (auto-converted to JSON Schema). Both SDKs surface the evidence field.
  • G5 (No Breaking Changes): All additions are optional fields with defaults that preserve current behavior. Existing callers are unaffected.

Non-Goals

  • Inline citations: Evidence is a separate field on the response, not inline markers within content. Inline citation formatting is presentation-layer and out of scope.
  • Tool calling via the OpenAI-compatible endpoint: The /v1/chat/completions endpoint does not expose Honcho’s internal tools to the caller. It is a completions-only interface.
  • Function calling passthrough: The OpenAI-compatible endpoint does not forward OpenAI-style function/tool definitions to the Dialectic agent.
  • Modifying the agentic tool loop: The agent continues to run identically. Structured output only affects the final synthesis call. Evidence is collected passively from existing tool_calls_made data.
  • Database migrations: No schema changes are required. All new data flows through request/response bodies.
  • Reasoning trace integration in evidence: If the reasoning traces spec (separate) is implemented, evidence can reference trace IDs. This spec designs the field to accommodate that but does not depend on it.

3. Design

3.A Structured Outputs

3.A.1 API Contract

Add an optional response_format field to DialecticOptions:

# src/schemas/api.py
 
class DialecticOptions(BaseModel):
    session_id: str | None = Field(None, ...)
    target: str | None = Field(None, ...)
    query: Annotated[str, Field(min_length=1, max_length=10000, ...)]
    stream: bool = False
    reasoning_level: ReasoningLevel = Field(default="low", ...)
    # NEW
    response_format: dict[str, Any] | None = Field(
        default=None,
        description=(
            "Optional JSON Schema describing the desired output structure. "
            "When provided, the final synthesis is formatted to conform to this schema. "
            "The schema must be a valid JSON Schema object (draft 2020-12 or earlier). "
            "The response content will be a JSON string matching this schema."
        ),
    )

Request example (non-streaming):

POST /v3/workspaces/my-app/peers/user-123/chat
{
  "query": "What are this user's top 3 food preferences?",
  "session_id": "session-abc",
  "reasoning_level": "low",
  "response_format": {
    "type": "object",
    "properties": {
      "preferences": {
        "type": "array",
        "items": {
          "type": "object",
          "properties": {
            "food": { "type": "string" },
            "sentiment": { "type": "string", "enum": ["loves", "likes", "neutral", "dislikes", "hates"] },
            "confidence": { "type": "number", "minimum": 0, "maximum": 1 }
          },
          "required": ["food", "sentiment"]
        },
        "maxItems": 3
      },
      "summary": { "type": "string" }
    },
    "required": ["preferences", "summary"]
  }
}

Response example:

{
  "content": "{\"preferences\":[{\"food\":\"dark roast coffee\",\"sentiment\":\"loves\",\"confidence\":0.95},{\"food\":\"sourdough bread\",\"sentiment\":\"likes\",\"confidence\":0.8},{\"food\":\"sushi\",\"sentiment\":\"likes\",\"confidence\":0.7}],\"summary\":\"The user has strong preferences for dark roast coffee and enjoys artisan breads and Japanese cuisine.\"}",
  "evidence": null
}

content is always a string. When response_format is provided, the string is a JSON-encoded object conforming to the schema. This preserves the content: str | None type contract and avoids polymorphic response types.

Streaming behavior: When stream: true and response_format is set, the stream emits the JSON string character-by-character in delta.content chunks, identical to how plain text streams today. The final accumulated string is valid JSON. Structured output streaming is supported because providers stream structured output tokens natively.

3.A.2 JSON Schema to Pydantic Conversion

The server must convert the caller’s JSON Schema into a Pydantic model at request time so it can be passed as response_model to honcho_llm_call(). This is a well-understood pattern:

# src/utils/schema_conversion.py (new file)
 
import json
from typing import Any
 
from pydantic import BaseModel, Field, create_model
 
 
def json_schema_to_pydantic(schema: dict[str, Any]) -> type[BaseModel]:
    """
    Convert a JSON Schema object into a dynamic Pydantic model.
 
    Supports:
    - Primitive types: string, number, integer, boolean
    - Objects with properties (nested recursively)
    - Arrays with items (typed lists)
    - Enums (via Literal)
    - Required fields
    - Default values
    - Description -> Field(description=...)
 
    Raises ValueError for unsupported schema constructs.
    """
    ...

Conversion rules:

JSON Schema TypePydantic Type
"string"str
"number"float
"integer"int
"boolean"bool
"array" with itemslist[T] where T is recursive
"object" with propertiesNested BaseModel subclass
"enum"Literal[...]
"null"None
anyOf / oneOf with nullOptional[T]

Unsupported constructs (return 422 Validation Error):

  • $ref / $defs (callers must inline their schemas)
  • allOf / not / if / then / else
  • patternProperties / additionalProperties with schemas
  • minItems / maxItems / minLength / maxLength (validated by the model, not by Pydantic — these are hints to the LLM)

Validation: The server validates the provided response_format is a syntactically valid JSON Schema object with "type": "object" at the root. Non-object root types are rejected with 422 because the LLM structured output constraint requires an object root.

Caching: Dynamic model creation is lightweight (microseconds) and does not need caching. Each request creates a fresh model class.

3.A.3 Integration into the Agent Pipeline

The structured output applies only to the final synthesis call, not to the tool loop iterations. The agent reasons in free-form text during tool use and only formats its answer at the end.

Non-streaming path (DialecticAgent.answer()):

# src/dialectic/core.py -- answer() method changes
 
async def answer(self, query: str, response_model: type[BaseModel] | None = None) -> str:
    tool_executor, task_name, run_id, start_time = await self._prepare_query(query)
    level_settings = settings.DIALECTIC.LEVELS[self.reasoning_level]
    tools = (
        DIALECTIC_TOOLS_MINIMAL if self.reasoning_level == "minimal"
        else DIALECTIC_TOOLS
    )
    max_tokens = (
        level_settings.MAX_OUTPUT_TOKENS
        if level_settings.MAX_OUTPUT_TOKENS is not None
        else settings.DIALECTIC.MAX_OUTPUT_TOKENS
    )
 
    response: HonchoLLMCallResponse[Any] = await honcho_llm_call(
        llm_settings=level_settings,
        prompt="",
        max_tokens=max_tokens,
        tools=tools,
        tool_choice=level_settings.TOOL_CHOICE,
        tool_executor=tool_executor,
        max_tool_iterations=level_settings.MAX_TOOL_ITERATIONS,
        messages=self.messages,
        track_name="Dialectic Agent",
        thinking_budget_tokens=level_settings.THINKING_BUDGET_TOKENS,
        max_input_tokens=settings.DIALECTIC.MAX_INPUT_TOKENS,
        trace_name="dialectic_chat",
        response_model=response_model,  # NEW: passed through to final synthesis call
    )
 
    # When response_model is set, response.content is the Pydantic model instance.
    # Serialize it to a JSON string for the API response.
    content = response.content
    if response_model is not None and isinstance(content, BaseModel):
        content = content.model_dump_json()
 
    self._log_response_metrics(...)
    return content

How honcho_llm_call() handles this: The tool execution loop in honcho_llm_call() already supports response_model. During tool iterations, the model returns tool calls (not structured output). On the final call (when no tool calls are made, or max iterations reached), response_model is passed to honcho_llm_call_inner(), which instructs the provider to return structured output. This is the existing behavior — no changes to clients.py are needed.

Streaming path (DialecticAgent.answer_stream()):

When stream=True and response_model is set, the stream_final_only=True flag is already set by the current code. The tool loop runs non-streaming, and the final response is streamed. The _stream_final_response() function already receives response_model and passes it through to the provider. The stream emits JSON tokens that accumulate into a valid JSON string.

The answer_stream() method gains the same response_model parameter:

async def answer_stream(
    self, query: str, response_model: type[BaseModel] | None = None
) -> AsyncIterator[str]:
    # ... same setup ...
    response = cast(
        StreamingResponseWithMetadata,
        await honcho_llm_call(
            ...,
            stream=True,
            stream_final_only=True,
            response_model=response_model,  # NEW
            ...,
        ),
    )
    # ... same streaming logic ...

3.A.4 Error Handling

ConditionHTTP StatusError
response_format is not a valid JSON Schema422Pydantic validation error on DialecticOptions
response_format root type is not "object"422"response_format must have root type 'object'"
response_format uses unsupported constructs ($ref, etc.)422"response_format contains unsupported JSON Schema construct: $ref"
LLM fails to produce valid structured output after retries500Standard LLMError, same as any LLM failure

Validation of response_format happens eagerly in the router before the agent is created, so malformed schemas fail fast.

3.A.5 SDK Integration

Python SDK:

from pydantic import BaseModel
from honcho import Honcho
 
class FoodPreferences(BaseModel):
    preferences: list[dict]
    summary: str
 
honcho = Honcho(workspace_id="my-app")
peer = honcho.peer("user-123")
 
# Pass a Pydantic model -- SDK converts to JSON Schema via .model_json_schema()
result = peer.chat(
    "What are this user's food preferences?",
    response_format=FoodPreferences,
)
# result is a FoodPreferences instance (SDK parses the JSON content)
print(result.preferences)
 
# Or pass raw JSON Schema dict (no parsing, content is a JSON string)
result_raw = peer.chat(
    "What are this user's food preferences?",
    response_format={"type": "object", "properties": {"items": {"type": "array", "items": {"type": "string"}}}},
)
# result_raw is str (JSON string)

Implementation in peer.py:

@validate_call(config=ConfigDict(arbitrary_types_allowed=True))
def chat(
    self,
    query: str = Field(..., min_length=1),
    *,
    target: str | PeerBase | None = None,
    session: str | SessionBase | None = None,
    reasoning_level: Literal["minimal", "low", "medium", "high", "max"] | None = None,
    response_format: type[BaseModel] | dict[str, Any] | None = None,  # NEW
) -> str | None:
    # ... existing setup ...
 
    body: dict[str, Any] = {"query": query, "stream": False}
    # ... existing target/session/reasoning_level handling ...
 
    # Convert Pydantic model to JSON Schema if needed
    response_format_schema: dict[str, Any] | None = None
    if response_format is not None:
        if isinstance(response_format, type) and issubclass(response_format, BaseModel):
            response_format_schema = response_format.model_json_schema()
        else:
            response_format_schema = response_format
        body["response_format"] = response_format_schema
 
    data = self._honcho._http.post(
        routes.peer_chat(self.workspace_id, self.id),
        body=body,
    )
    content = data.get("content")
    if content is None:
        return None
 
    # If a Pydantic model was passed, parse the JSON content into the model
    if isinstance(response_format, type) and issubclass(response_format, BaseModel):
        return response_format.model_validate_json(content)
 
    return content

The return type becomes str | BaseModel | None. For type safety, the SDK can use @overload to provide precise return types based on whether response_format is a type[M] or a dict.

TypeScript SDK:

import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
 
const FoodPreferences = z.object({
  preferences: z.array(z.object({
    food: z.string(),
    sentiment: z.enum(["loves", "likes", "neutral", "dislikes", "hates"]),
  })),
  summary: z.string(),
});
 
const result = await peer.chat("What are this user's food preferences?", {
  responseFormat: FoodPreferences,
});
// result is typed as z.infer<typeof FoodPreferences>

The TypeScript SDK detects Zod schemas via duck-typing (value._def !== undefined), converts to JSON Schema using zod-to-json-schema, sends the schema to the API, and parses the response JSON with schema.parse().

zod-to-json-schema is added as a peer dependency.

3.B Evidence / Citations

3.B.1 API Contract

Add an optional include_evidence field to DialecticOptions and an evidence field to DialecticResponse:

# src/schemas/api.py
 
class DialecticOptions(BaseModel):
    # ... existing fields ...
    include_evidence: bool = Field(
        default=False,
        description=(
            "When true, the response includes an 'evidence' field containing "
            "the conclusion IDs, message IDs, and observations accessed during "
            "the agent's tool loop."
        ),
    )
 
 
class EvidenceObservation(BaseModel):
    """An observation accessed during the dialectic tool loop."""
    id: str = Field(description="Document/conclusion ID")
    content: str = Field(description="The observation content")
    level: str = Field(description="Observation level: explicit, deductive, inductive, contradiction")
    created_at: datetime.datetime = Field(description="When the observation was created")
    source_ids: list[str] | None = Field(
        default=None,
        description="IDs of premise/source observations (for deductive/inductive/contradiction)",
    )
    session_id: str | None = Field(
        default=None,
        description="Session the observation is scoped to, if any",
    )
 
 
class EvidenceMessageRef(BaseModel):
    """A message accessed during the dialectic tool loop."""
    id: str = Field(description="Message public ID")
    session_id: str = Field(description="Session the message belongs to")
    peer_id: str = Field(description="Peer who sent the message")
    content_preview: str = Field(
        description="First 200 characters of the message content"
    )
    created_at: datetime.datetime = Field(description="When the message was sent")
 
 
class Evidence(BaseModel):
    """Evidence gathered during the dialectic agent's tool loop."""
    conclusions: list[EvidenceObservation] = Field(
        default_factory=list,
        description="Observations/conclusions the agent accessed via search_memory, get_reasoning_chain, etc.",
    )
    messages: list[EvidenceMessageRef] = Field(
        default_factory=list,
        description="Messages the agent accessed via search_messages, grep_messages, etc.",
    )
    tool_calls: list[dict[str, Any]] = Field(
        default_factory=list,
        description=(
            "Raw tool call log: [{tool_name, tool_input}] for every tool invocation. "
            "Does not include tool results (which can be large)."
        ),
    )
    reasoning_trace_id: str | None = Field(
        default=None,
        description="ID of the reasoning trace for this dialectic call, if reasoning trace storage is enabled.",
    )
 
 
class DialecticResponse(BaseModel):
    content: str | None
    evidence: Evidence | None = Field(
        default=None,
        description="Evidence supporting the response. Only present when include_evidence=true in the request.",
    )

Response example with evidence:

POST /v3/workspaces/my-app/peers/user-123/chat
{
  "query": "What coffee does this user prefer?",
  "include_evidence": true
}
{
  "content": "The user prefers dark roast coffee, specifically from local roasters.",
  "evidence": {
    "conclusions": [
      {
        "id": "doc_abc123",
        "content": "User prefers dark roast coffee",
        "level": "explicit",
        "created_at": "2026-03-20T10:15:00Z",
        "source_ids": null,
        "session_id": "session-xyz"
      },
      {
        "id": "doc_def456",
        "content": "User tends to buy from local roasters based on multiple mentions of 'Blue Bottle' and 'Stumptown'",
        "level": "inductive",
        "created_at": "2026-03-22T14:30:00Z",
        "source_ids": ["doc_abc123", "doc_ghi789"],
        "session_id": null
      }
    ],
    "messages": [
      {
        "id": "msg_m1n2o3",
        "session_id": "session-xyz",
        "peer_id": "user-123",
        "content_preview": "I just picked up some amazing dark roast from Blue Bottle. It's so much better than...",
        "created_at": "2026-03-20T10:12:00Z"
      }
    ],
    "tool_calls": [
      {"tool_name": "search_memory", "tool_input": {"query": "coffee preference", "top_k": 20}},
      {"tool_name": "search_messages", "tool_input": {"query": "coffee roast preference"}}
    ],
    "reasoning_trace_id": null
  }
}

When include_evidence is false (default): The evidence field is null. This is the current behavior — zero overhead for callers who do not need evidence.

3.B.2 Evidence Collection Architecture

Evidence is collected passively from data that already flows through the tool execution pipeline. The model is never asked to produce citations. This keeps model load at zero and makes evidence collection deterministic.

Collection points:

Data SourceWhat is CollectedWhere in Code
tool_calls_made on HonchoLLMCallResponse / StreamingResponseWithMetadataTool names and inputs for the tool_calls listAlready captured in honcho_llm_call() tool loop
_handle_search_memory tool resultConclusion IDs and observation objects from Representation.from_documents()New: tool handler returns structured data alongside string result
_handle_search_messages / _handle_grep_messages / _handle_search_messages_temporal / _handle_get_messages_by_date_range tool resultsMessage IDs from matched models.Message objectsNew: tool handler returns structured data alongside string result
_handle_get_observation_context tool resultMessage IDs from context messagesNew: tool handler returns structured data alongside string result
_handle_get_reasoning_chain tool resultConclusion IDs from premise/conclusion traversalNew: tool handler returns structured data alongside string result
_prefetch_relevant_observationsConclusion IDs from prefetched observationsNew: agent tracks prefetched document IDs

Implementation approach — ToolContext evidence accumulator:

Add an evidence accumulator to ToolContext that tool handlers append to:

# src/utils/agent_tools.py
 
@dataclass
class EvidenceAccumulator:
    """Collects evidence during tool execution for the dialectic agent."""
    conclusion_ids: set[str] = field(default_factory=set)
    conclusions: dict[str, models.Document] = field(default_factory=dict)  # id -> Document
    message_ids: set[str] = field(default_factory=set)
    messages: dict[str, models.Message] = field(default_factory=dict)  # public_id -> Message
 
    def add_conclusions(self, documents: Sequence[models.Document]) -> None:
        """Record conclusions accessed by a tool."""
        for doc in documents:
            if doc.id not in self.conclusion_ids:
                self.conclusion_ids.add(doc.id)
                self.conclusions[doc.id] = doc
 
    def add_messages(self, msgs: Sequence[models.Message]) -> None:
        """Record messages accessed by a tool."""
        for msg in msgs:
            if msg.public_id not in self.message_ids:
                self.message_ids.add(msg.public_id)
                self.messages[msg.public_id] = msg
 
 
@dataclass
class ToolContext:
    """Context object passed to tool handlers."""
    db: AsyncSession
    workspace_name: str
    observer: str
    observed: str
    session_name: str | None
    current_messages: list[models.Message] | None
    include_observation_ids: bool
    history_token_limit: int
    db_lock: asyncio.Lock
    configuration: ResolvedConfiguration | None = None
    run_id: str | None = None
    agent_type: str | None = None
    parent_category: str | None = None
    evidence: EvidenceAccumulator | None = None  # NEW: only created when include_evidence=True

Tool handler changes (example for _handle_search_memory):

async def _handle_search_memory(ctx: ToolContext, tool_input: dict[str, Any]) -> str:
    top_k = min(_safe_int(tool_input.get("top_k"), 20), 40)
    query = tool_input["query"]
    # ... existing embedding + query_documents logic ...
 
    documents = await crud.query_documents(...)
    mem = Representation.from_documents(documents)
 
    # NEW: record accessed conclusions in evidence accumulator
    if ctx.evidence is not None:
        ctx.evidence.add_conclusions(documents)
 
    # ... existing formatting and return ...

The same pattern applies to every read tool:

  • _handle_search_messages: calls ctx.evidence.add_messages(...) with matched messages
  • _handle_grep_messages: calls ctx.evidence.add_messages(...) with matched messages
  • _handle_get_messages_by_date_range: calls ctx.evidence.add_messages(...) with messages
  • _handle_search_messages_temporal: calls ctx.evidence.add_messages(...) with matched messages
  • _handle_get_observation_context: calls ctx.evidence.add_messages(...) with context messages
  • _handle_get_reasoning_chain: calls ctx.evidence.add_conclusions(...) with traversed documents

Write tools (create_observations, update_peer_card, delete_observations) do not contribute to evidence — evidence captures what the agent read, not what it wrote.

3.B.3 Plumbing Evidence Through the Pipeline

The evidence accumulator needs to flow from ToolContext through the agent back to the router.

create_tool_executor changes:

async def create_tool_executor(
    ...,
    include_evidence: bool = False,  # NEW
) -> tuple[Callable[[str, dict[str, Any]], Any], EvidenceAccumulator | None]:  # NEW: return type changes
    """..."""
    shared_lock = await get_observation_lock(workspace_name, observer, observed)
 
    evidence = EvidenceAccumulator() if include_evidence else None
 
    ctx = ToolContext(
        ...,
        evidence=evidence,  # NEW
    )
 
    async def execute_tool(tool_name: str, tool_input: dict[str, Any]) -> str:
        # ... existing logic, unchanged ...
 
    return execute_tool, evidence  # NEW: return evidence accumulator

DialecticAgent changes:

class DialecticAgent:
    def __init__(self, ..., include_evidence: bool = False):
        # ... existing init ...
        self.include_evidence = include_evidence
        self._evidence: EvidenceAccumulator | None = None
 
    async def _prepare_query(self, query: str) -> tuple[...]:
        # ... existing logic ...
 
        tool_executor, evidence = await create_tool_executor(
            ...,
            include_evidence=self.include_evidence,  # NEW
        )
        self._evidence = evidence
 
        # Record prefetched conclusions in evidence
        if evidence is not None and prefetched_observations:
            # The prefetch step already queried documents -- add them
            # (requires _prefetch_relevant_observations to return documents alongside text)
            pass  # See 3.B.4
 
        return tool_executor, task_name, run_id, start_time
 
    async def answer(self, query: str, response_model: type[BaseModel] | None = None) -> tuple[str, Evidence | None]:
        # ... existing logic ...
        content = response.content
        # ... serialize if response_model ...
 
        evidence = self._build_evidence(response) if self.include_evidence else None
        return content, evidence
 
    def _build_evidence(self, response: HonchoLLMCallResponse) -> Evidence:
        """Build Evidence object from accumulated tool data."""
        from src.schemas.api import Evidence, EvidenceObservation, EvidenceMessageRef
 
        conclusions = []
        if self._evidence:
            for doc in self._evidence.conclusions.values():
                conclusions.append(EvidenceObservation(
                    id=doc.id,
                    content=doc.content,
                    level=doc.level,
                    created_at=doc.created_at,
                    source_ids=doc.source_ids,
                    session_id=doc.session_name,
                ))
 
        messages = []
        if self._evidence:
            for msg in self._evidence.messages.values():
                messages.append(EvidenceMessageRef(
                    id=msg.public_id,
                    session_id=msg.session_name,
                    peer_id=msg.peer_name,
                    content_preview=msg.content[:200],
                    created_at=msg.created_at,
                ))
 
        tool_calls = [
            {"tool_name": tc.get("tool_name", tc.get("name", "")),
             "tool_input": tc.get("tool_input", tc.get("input", {}))}
            for tc in response.tool_calls_made
        ]
 
        return Evidence(
            conclusions=conclusions,
            messages=messages,
            tool_calls=tool_calls,
            reasoning_trace_id=None,  # Populated when reasoning traces spec is implemented
        )

3.B.4 Prefetched Observation Evidence

_prefetch_relevant_observations currently calls search_memory() which returns a Representation. To capture the underlying Document objects for evidence, the method needs the raw documents:

async def _prefetch_relevant_observations(self, query: str) -> str | None:
    # ... existing embedding logic ...
 
    explicit_docs = await crud.query_documents(
        db=self.db, workspace_name=self.workspace_name,
        observer=self.observer, observed=self.observed,
        query=query, top_k=prefetch_limit,
        filters={"level": {"in": ["explicit"]}},
        embedding=query_embedding,
    )
    derived_docs = await crud.query_documents(
        db=self.db, workspace_name=self.workspace_name,
        observer=self.observer, observed=self.observed,
        query=query, top_k=prefetch_limit,
        filters={"level": {"in": ["deductive", "inductive", "contradiction"]}},
        embedding=query_embedding,
    )
 
    # Record in evidence accumulator
    if self._evidence is not None:
        self._evidence.add_conclusions(explicit_docs)
        self._evidence.add_conclusions(derived_docs)
 
    explicit_repr = Representation.from_documents(explicit_docs)
    derived_repr = Representation.from_documents(derived_docs)
    # ... existing formatting logic ...

This is a minor refactor: instead of calling the search_memory helper (which wraps crud.query_documents + Representation.from_documents), we call crud.query_documents directly and then build the Representation. The existing search_memory helper remains unchanged for use by tool handlers.

3.B.5 Streaming Evidence

For streaming responses, evidence must be delivered after the stream completes. Two options:

Option A (chosen): Final SSE event with evidence.

After the content stream completes, emit one additional SSE event containing the evidence:

data: {"delta": {"content": "The user prefers"}, "done": false}
data: {"delta": {"content": " dark roast coffee."}, "done": false}
data: {"done": true, "evidence": {"conclusions": [...], "messages": [...], "tool_calls": [...], "reasoning_trace_id": null}}

This extends DialecticStreamChunk:

class DialecticStreamChunk(BaseModel):
    delta: DialecticStreamDelta
    done: bool = False
    evidence: Evidence | None = None  # NEW: only set on the final chunk when done=true

The format_sse_stream function in peers.py is updated:

async def format_sse_stream(
    chunks: AsyncIterator[str],
    evidence_fn: Callable[[], Evidence | None] | None = None,
) -> AsyncIterator[str]:
    async for chunk in chunks:
        yield f"data: {json.dumps({'delta': {'content': chunk}, 'done': False})}\n\n"
    # Final event
    final: dict[str, Any] = {"done": True}
    if evidence_fn is not None:
        ev = evidence_fn()
        if ev is not None:
            final["evidence"] = json.loads(ev.model_dump_json())
    yield f"data: {json.dumps(final)}\n\n"

Why not Option B (separate endpoint)? Requiring a second request to fetch evidence defeats the purpose. Evidence should be available inline with the response.

3.B.6 Evidence Size Limits

Evidence can be large if the agent makes many tool calls. To prevent response bloat:

  • conclusions: No limit. The agent typically accesses 10-50 conclusions. Full observation objects are included per the requirements.
  • messages: content_preview is capped at 200 characters. Full message content is not included.
  • tool_calls: Tool results are excluded (only tool_name and tool_input). Tool inputs are small (query strings, IDs). This keeps the list compact.
  • Total evidence payload: If evidence exceeds 100KB when serialized, truncate tool_calls to the last 20 entries and add a truncated: true flag. This is a safety valve, not expected in practice.

3.B.7 SDK Integration

Python SDK:

from honcho.api_types import Evidence
 
result = peer.chat("What coffee does the user like?", include_evidence=True)
# result is a ChatResponse with .content and .evidence
 
print(result.content)
# "The user prefers dark roast coffee."
 
print(result.evidence.conclusions[0].content)
# "User prefers dark roast coffee"

When include_evidence=True, the chat() method returns a ChatResponse object (new type) instead of str | None:

class ChatResponse(BaseModel):
    content: str | None
    evidence: Evidence | None
 
class Evidence(BaseModel):
    conclusions: list[EvidenceObservation]
    messages: list[EvidenceMessageRef]
    tool_calls: list[dict[str, Any]]
    reasoning_trace_id: str | None

For chat_stream() with evidence, the stream yields content chunks as before, and the DialecticStreamResponse object gains an .evidence property populated from the final SSE event.

TypeScript SDK:

Same pattern. chat() returns ChatResponse when includeEvidence: true is set, containing typed Evidence object.

3.C OpenAI-Compatible Completions Endpoint

3.C.1 Route and Authentication

POST /v1/chat/completions

This route is mounted at the /v1 prefix (not /v3) to match the OpenAI convention. It is added as a separate router in src/routers/completions.py.

Authentication: Standard Honcho JWT via Authorization: Bearer <token>, same as all other endpoints. The JWT must have workspace-level or broader access.

Model string: The model field in the OpenAI request encodes the Honcho workspace and peer:

workspace/{workspace_id}/peer/{peer_id}

Example: "model": "workspace/my-app/peer/user-123"

Session: Provided via the X-Honcho-Session-Id header (optional). When omitted, the Dialectic runs without session scoping (global query).

Target peer: Provided via the X-Honcho-Target header (optional). When omitted, observer == observed (omniscient Honcho perspective).

3.C.2 Request Mapping

OpenAI Chat Completion request:

{
  "model": "workspace/my-app/peer/user-123",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What does this user like to eat?"}
  ],
  "stream": true,
  "temperature": 0.7,
  "max_tokens": 1024,
  "response_format": { "type": "json_schema", "json_schema": { "name": "food_prefs", "schema": { "type": "object", "properties": { "items": { "type": "array", "items": { "type": "string" } } } } } }
}

Mapping rules:

OpenAI FieldHoncho Mapping
modelParsed to extract workspace_id and peer_id
messagesThe last user message becomes DialecticOptions.query. System messages and earlier user messages are ignored (the Dialectic has its own system prompt and uses Honcho’s memory, not the chat history).
streamMaps to DialecticOptions.stream
temperatureIgnored (Dialectic uses its configured temperature)
max_tokensIgnored (Dialectic uses its configured max tokens)
response_formatWhen type is "json_schema", the json_schema.schema object maps to DialecticOptions.response_format. When type is "json_object", ignored (no schema constraint). When type is "text", no structured output.
nMust be 1 or omitted. Values > 1 return 400.
tools / tool_choiceIgnored. Returns 400 if tools is non-empty (tool calling not supported).
top_p / frequency_penalty / presence_penaltyIgnored
stopIgnored

Reasoning level selection: An additional non-standard field honcho_reasoning_level can be included in the request body. When omitted, defaults to "low". Alternatively, this can be set via the X-Honcho-Reasoning-Level header.

3.C.3 Response Mapping

Non-streaming response:

Honcho DialecticResponse:

{"content": "The user enjoys sushi and dark roast coffee."}

OpenAI-format response:

{
  "id": "chatcmpl-honcho-abc123",
  "object": "chat.completion",
  "created": 1711324800,
  "model": "workspace/my-app/peer/user-123",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The user enjoys sushi and dark roast coffee."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

usage is zeroed out because Honcho does not expose internal token counts to callers. Providing fake numbers would be misleading. The id is generated as "chatcmpl-honcho-" + nanoid.

Streaming response (SSE):

data: {"id":"chatcmpl-honcho-abc123","object":"chat.completion.chunk","created":1711324800,"model":"workspace/my-app/peer/user-123","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-honcho-abc123","object":"chat.completion.chunk","created":1711324800,"model":"workspace/my-app/peer/user-123","choices":[{"index":0,"delta":{"content":"The user"},"finish_reason":null}]}

data: {"id":"chatcmpl-honcho-abc123","object":"chat.completion.chunk","created":1711324800,"model":"workspace/my-app/peer/user-123","choices":[{"index":0,"delta":{"content":" enjoys sushi."},"finish_reason":null}]}

data: {"id":"chatcmpl-honcho-abc123","object":"chat.completion.chunk","created":1711324800,"model":"workspace/my-app/peer/user-123","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

This exactly matches the OpenAI SSE format, including the [DONE] sentinel. The openai Python and TypeScript clients can consume this stream without modification.

3.C.4 Implementation

# src/routers/completions.py (new file)
 
import json
import logging
import re
import time
from collections.abc import AsyncIterator
from typing import Any
 
from fastapi import APIRouter, Body, Header, Request
from fastapi.responses import StreamingResponse
 
from src import schemas
from src.dialectic.chat import agentic_chat, agentic_chat_stream
from src.exceptions import ValidationException
from src.security import require_auth
from src.utils.tokens import generate_nanoid
 
logger = logging.getLogger(__name__)
 
router = APIRouter(prefix="/v1", tags=["openai-compat"])
 
MODEL_PATTERN = re.compile(r"^workspace/(?P<workspace_id>[^/]+)/peer/(?P<peer_id>[^/]+)$")
 
 
def _parse_model_string(model: str) -> tuple[str, str]:
    """Parse 'workspace/{id}/peer/{id}' into (workspace_id, peer_id)."""
    match = MODEL_PATTERN.match(model)
    if not match:
        raise ValidationException(
            f"Invalid model string: '{model}'. "
            "Expected format: 'workspace/{{workspace_id}}/peer/{{peer_id}}'"
        )
    return match.group("workspace_id"), match.group("peer_id")
 
 
def _extract_query(messages: list[dict[str, Any]]) -> str:
    """Extract the query from the last user message."""
    for msg in reversed(messages):
        if msg.get("role") == "user":
            content = msg.get("content", "")
            if isinstance(content, str):
                return content
            # Handle content array (OpenAI vision format) -- extract text parts
            if isinstance(content, list):
                texts = [
                    part["text"] for part in content
                    if isinstance(part, dict) and part.get("type") == "text"
                ]
                return " ".join(texts)
    raise ValidationException("No user message found in messages array")
 
 
def _extract_response_format(body: dict[str, Any]) -> dict[str, Any] | None:
    """Extract JSON Schema from OpenAI response_format if present."""
    rf = body.get("response_format")
    if rf is None:
        return None
    rf_type = rf.get("type", "text")
    if rf_type == "json_schema":
        schema_obj = rf.get("json_schema", {})
        return schema_obj.get("schema")
    return None
 
 
def _make_completion_id() -> str:
    return f"chatcmpl-honcho-{generate_nanoid()}"
 
 
@router.post("/chat/completions")
async def chat_completions(
    request: Request,
    body: dict[str, Any] = Body(...),
    x_honcho_session_id: str | None = Header(None, alias="X-Honcho-Session-Id"),
    x_honcho_target: str | None = Header(None, alias="X-Honcho-Target"),
    x_honcho_reasoning_level: str | None = Header(None, alias="X-Honcho-Reasoning-Level"),
):
    """OpenAI-compatible chat completions endpoint backed by Honcho Dialectic."""
    # Parse model string
    model_str = body.get("model", "")
    workspace_id, peer_id = _parse_model_string(model_str)
 
    # Validate authentication (reuse Honcho auth)
    # The JWT is in the Authorization header, standard Honcho auth
    # We validate workspace access
    await require_auth(workspace_name=workspace_id)(request)
 
    # Extract query from messages
    messages = body.get("messages", [])
    if not messages:
        raise ValidationException("messages array is required and must not be empty")
    query = _extract_query(messages)
 
    # Reject unsupported features
    if body.get("tools"):
        raise ValidationException("Tool calling is not supported on this endpoint")
    n = body.get("n", 1)
    if n != 1:
        raise ValidationException("Only n=1 is supported")
 
    # Map parameters
    stream = body.get("stream", False)
    response_format = _extract_response_format(body)
    reasoning_level = x_honcho_reasoning_level or body.get("honcho_reasoning_level", "low")
 
    # Build completion ID and timestamp
    completion_id = _make_completion_id()
    created = int(time.time())
 
    if stream:
        async def openai_sse_stream() -> AsyncIterator[str]:
            # First chunk: role
            first_chunk = {
                "id": completion_id,
                "object": "chat.completion.chunk",
                "created": created,
                "model": model_str,
                "choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": None}],
            }
            yield f"data: {json.dumps(first_chunk)}\n\n"
 
            # Content chunks
            async for text_chunk in agentic_chat_stream(
                workspace_name=workspace_id,
                session_name=x_honcho_session_id,
                query=query,
                observer=peer_id,
                observed=x_honcho_target if x_honcho_target else peer_id,
                reasoning_level=reasoning_level,
            ):
                chunk = {
                    "id": completion_id,
                    "object": "chat.completion.chunk",
                    "created": created,
                    "model": model_str,
                    "choices": [{"index": 0, "delta": {"content": text_chunk}, "finish_reason": None}],
                }
                yield f"data: {json.dumps(chunk)}\n\n"
 
            # Final chunk: finish_reason
            final_chunk = {
                "id": completion_id,
                "object": "chat.completion.chunk",
                "created": created,
                "model": model_str,
                "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
            }
            yield f"data: {json.dumps(final_chunk)}\n\n"
            yield "data: [DONE]\n\n"
 
        return StreamingResponse(
            openai_sse_stream(),
            media_type="text/event-stream",
            headers={"Cache-Control": "no-cache", "Connection": "keep-alive"},
        )
 
    # Non-streaming
    content = await agentic_chat(
        workspace_name=workspace_id,
        session_name=x_honcho_session_id,
        query=query,
        observer=peer_id,
        observed=x_honcho_target if x_honcho_target else peer_id,
        reasoning_level=reasoning_level,
    )
 
    return {
        "id": completion_id,
        "object": "chat.completion",
        "created": created,
        "model": model_str,
        "choices": [
            {
                "index": 0,
                "message": {"role": "assistant", "content": content},
                "finish_reason": "stop",
            }
        ],
        "usage": {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0},
    }

Router registration in src/main.py:

from src.routers import completions
 
# After existing router registrations:
app.include_router(completions.router)  # No prefix -- router has /v1 prefix built in

3.C.5 Client Compatibility

The endpoint is designed to work as a drop-in with:

# openai-python
from openai import OpenAI
 
client = OpenAI(
    base_url="https://api.honcho.dev",
    api_key="<honcho-jwt>",
)
 
response = client.chat.completions.create(
    model="workspace/my-app/peer/user-123",
    messages=[{"role": "user", "content": "What does this user like?"}],
    extra_headers={"X-Honcho-Session-Id": "session-abc"},
)
print(response.choices[0].message.content)
// openai-typescript
import OpenAI from "openai";
 
const client = new OpenAI({
  baseURL: "https://api.honcho.dev",
  apiKey: "<honcho-jwt>",
});
 
const stream = await client.chat.completions.create({
  model: "workspace/my-app/peer/user-123",
  messages: [{ role: "user", content: "What does this user like?" }],
  stream: true,
}, {
  headers: { "X-Honcho-Session-Id": "session-abc" },
});
 
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}

3.C.6 Structured Output via OpenAI Endpoint

When the OpenAI request includes response_format: { type: "json_schema", json_schema: { name: "...", schema: {...} } }, the schema is extracted and passed through to the Dialectic as DialecticOptions.response_format. The response content field contains the JSON string, which is the standard OpenAI behavior for structured output.

3.C.7 What is Explicitly Not Supported

OpenAI FeatureStatusBehavior
Tool/function callingNot supported400 error if tools is non-empty
Multiple choices (n > 1)Not supported400 error
Vision (image inputs)Not supportedImage content parts are silently ignored
AudioNot supported400 error
LogprobsNot supportedIgnored, not returned
seedIgnoredNo deterministic generation
Fine-tuned modelsN/AModel string is Honcho-specific
Embeddings, moderations, etc.Not in scopeOnly /v1/chat/completions is implemented

4. Implementation Phases

Phase 1: Structured Outputs (Feature A)

Estimated effort: 2-3 days

  1. Create src/utils/schema_conversion.py with json_schema_to_pydantic().
  2. Add response_format field to DialecticOptions in src/schemas/api.py.
  3. Add validation in src/routers/peers.py chat endpoint: parse and validate response_format, convert to Pydantic model.
  4. Thread response_model parameter through agentic_chat() / agentic_chat_stream() in src/dialectic/chat.py.
  5. Thread response_model parameter through DialecticAgent.answer() / answer_stream() in src/dialectic/core.py.
  6. Serialize Pydantic model instance to JSON string in answer() return.
  7. Add tests: valid schema, nested schema, array schema, invalid schema (422), unsupported constructs (422), streaming with structured output.
  8. Update Python SDK peer.chat() to accept response_format (Pydantic model or dict).
  9. Update TypeScript SDK peer.chat() to accept responseFormat (Zod schema or object).

Phase 2: Evidence / Citations (Feature B)

Estimated effort: 3-4 days

  1. Add EvidenceAccumulator dataclass to src/utils/agent_tools.py.
  2. Add evidence field to ToolContext.
  3. Update create_tool_executor to accept include_evidence and return evidence accumulator.
  4. Instrument 7 read tool handlers to append to evidence accumulator.
  5. Add Evidence, EvidenceObservation, EvidenceMessageRef schemas to src/schemas/api.py.
  6. Add include_evidence field to DialecticOptions.
  7. Add evidence field to DialecticResponse and DialecticStreamChunk.
  8. Update DialecticAgent to accept include_evidence, build evidence from accumulator.
  9. Refactor _prefetch_relevant_observations to capture documents for evidence.
  10. Update agentic_chat() / agentic_chat_stream() in src/dialectic/chat.py to thread evidence.
  11. Update chat() router in src/routers/peers.py to pass include_evidence and attach evidence to response.
  12. Update streaming SSE formatter to include evidence in final event.
  13. Add tests: evidence with search_memory, evidence with search_messages, evidence with grep, evidence with multiple tools, streaming evidence, evidence off by default.
  14. Update Python SDK to expose Evidence type and include_evidence parameter.
  15. Update TypeScript SDK equivalently.

Phase 3: OpenAI-Compatible Endpoint (Feature C)

Estimated effort: 2-3 days

  1. Create src/routers/completions.py with the full endpoint implementation.
  2. Register router in src/main.py.
  3. Add model string parser with validation.
  4. Add query extraction from messages array.
  5. Implement non-streaming response mapping.
  6. Implement streaming SSE response mapping with OpenAI format.
  7. Wire response_format passthrough (depends on Phase 1).
  8. Add tests: non-streaming, streaming, model string parsing, invalid model string, missing messages, tool calling rejection, session header, target header, reasoning level header.
  9. Add integration test with openai Python client.
  10. Document endpoint in API docs.

Phase ordering: Phase 1 and Phase 2 are independent and can be parallelized. Phase 3 depends on Phase 1 for response_format passthrough but can be started in parallel for the core endpoint without structured output support.

5. Files to Modify

New Files

FilePurpose
src/utils/schema_conversion.pyjson_schema_to_pydantic() conversion utility
src/routers/completions.pyOpenAI-compatible /v1/chat/completions endpoint
tests/test_schema_conversion.pyUnit tests for JSON Schema to Pydantic conversion
tests/test_completions.pyTests for the OpenAI-compatible endpoint
tests/test_dialectic_evidence.pyTests for evidence collection
tests/test_dialectic_structured.pyTests for structured output through the Dialectic

Modified Files

FileChanges
src/schemas/api.pyAdd response_format and include_evidence to DialecticOptions. Add evidence to DialecticResponse. Add evidence to DialecticStreamChunk. Add Evidence, EvidenceObservation, EvidenceMessageRef schemas.
src/dialectic/core.pyAdd response_model param to answer() and answer_stream(). Add include_evidence to __init__. Add _build_evidence() method. Refactor _prefetch_relevant_observations to capture documents. Return tuple (content, evidence) from answer().
src/dialectic/chat.pyThread response_model and include_evidence through agentic_chat() and agentic_chat_stream(). Return (content, evidence) tuple.
src/routers/peers.pyValidate and convert response_format to Pydantic model. Pass include_evidence through. Attach evidence to response. Update streaming SSE formatter for evidence in final event.
src/utils/agent_tools.pyAdd EvidenceAccumulator dataclass. Add evidence field to ToolContext. Update create_tool_executor to accept include_evidence and return evidence. Instrument 7 read tool handlers (_handle_search_memory, _handle_search_messages, _handle_grep_messages, _handle_get_messages_by_date_range, _handle_search_messages_temporal, _handle_get_observation_context, _handle_get_reasoning_chain).
src/main.pyRegister completions.router.
sdks/python/src/honcho/peer.pyAdd response_format and include_evidence params to chat() and chat_stream(). Add return type overloads.
sdks/python/src/honcho/aio.pyMirror changes from peer.py for async methods.
sdks/python/src/honcho/types.pyAdd ChatResponse, Evidence, EvidenceObservation, EvidenceMessageRef types.
sdks/python/src/honcho/api_types.pyAdd evidence-related API response types.
sdks/typescript/src/peer.tsAdd responseFormat and includeEvidence params. Add Zod-to-JSON-Schema conversion.
sdks/typescript/src/types/api.tsAdd Evidence, EvidenceObservation, EvidenceMessageRef types.

6. Risk Assessment

6.1 JSON Schema to Pydantic Conversion Edge Cases

Risk: Callers provide schemas that json_schema_to_pydantic() cannot handle, leading to unexpected 422 errors or runtime failures.

Mitigation: Start with a conservative set of supported constructs (primitives, objects, arrays, enums, required/optional). Reject unsupported constructs eagerly with clear error messages. Document the supported subset. Add a comprehensive test suite covering edge cases (deeply nested objects, arrays of arrays, enum arrays, empty objects, etc.).

Fallback: If conversion fails, fall back to json_mode=True (ask the LLM to produce JSON without schema constraint) and validate the output against the schema post-hoc. This is less reliable but provides graceful degradation.

6.2 Evidence Accumulator Memory Pressure

Risk: For queries with heavy tool use (max reasoning level, many iterations), the evidence accumulator could hold hundreds of Document and Message ORM objects in memory.

Mitigation: The accumulator uses a dict keyed by ID, so duplicates (same conclusion accessed by multiple tools) are stored once. Documents and Messages are already loaded into memory by the tool handlers — the accumulator just holds references, not copies. The content_preview truncation (200 chars) keeps the serialized evidence compact. In the worst case (50 conclusions + 100 messages), the serialized evidence is ~50KB — well within acceptable response size.

6.3 Streaming Structured Output Validity

Risk: If the stream is interrupted mid-JSON, the client receives invalid JSON.

Mitigation: This is inherent to streaming and matches the behavior of OpenAI’s structured output streaming. Clients that need guaranteed validity should use non-streaming mode. The SDKs should document this caveat.

6.4 OpenAI Client Compatibility Drift

Risk: Future versions of the openai Python/TypeScript clients may expect response fields that Honcho does not provide, breaking compatibility.

Mitigation: The endpoint returns the minimum viable OpenAI response shape. Fields like system_fingerprint are omitted — the openai client tolerates missing optional fields. Pin compatibility testing against openai>=1.0,<3.0 and run integration tests in CI.

6.5 Model String Parsing Ambiguity

Risk: Workspace or peer IDs containing / would break the workspace/{id}/peer/{id} model string format.

Mitigation: Honcho resource names are constrained to ^[a-zA-Z0-9_-]+$ (see RESOURCE_NAME_PATTERN in src/schemas/api.py). Slashes are never valid in workspace or peer names, so the parsing is unambiguous.

6.6 create_tool_executor Return Type Change

Risk: Changing create_tool_executor to return a tuple (executor, evidence) instead of just executor is a breaking change for all callers.

Mitigation: Two options:

  • Option A (preferred): Add the evidence accumulator as an attribute on ToolContext and expose it via a new method on the returned executor. The executor callable signature does not change. Callers that need evidence access it through the evidence accumulator reference returned by create_tool_executor. The return type becomes tuple[Callable, EvidenceAccumulator | None].

  • Option B: Keep the return type as just the callable, but attach evidence as an attribute on the function object. This is brittle and not recommended.

The Dialectic agent is the only caller that needs evidence. The Deriver and Dreamer callers can use _ to discard the second return value:

tool_executor, _ = await create_tool_executor(...)

7. Verification Plan

7.1 Unit Tests

Schema conversion (test_schema_conversion.py):

  • Primitive types (string, number, integer, boolean)
  • Nested objects (2 and 3 levels deep)
  • Arrays with typed items
  • Arrays of objects
  • Enum fields (string and integer)
  • Required vs optional fields
  • Default values
  • Unsupported constructs raise ValueError
  • Root type must be "object"
  • Empty properties object
  • description field propagation

Evidence accumulator (test_dialectic_evidence.py):

  • EvidenceAccumulator.add_conclusions() deduplicates by ID
  • EvidenceAccumulator.add_messages() deduplicates by public_id
  • _build_evidence() correctly serializes conclusions and messages
  • content_preview truncation at 200 characters
  • tool_calls extraction from HonchoLLMCallResponse.tool_calls_made
  • Evidence is None when include_evidence=False

OpenAI-compatible endpoint (test_completions.py):

  • Model string parsing: valid, invalid, missing workspace, missing peer
  • Query extraction: last user message, multiple user messages, content array format
  • Non-streaming response shape matches OpenAI spec
  • Streaming SSE format: first chunk has role, content chunks, final chunk has finish_reason, [DONE] sentinel
  • response_format passthrough
  • Rejection of tools parameter
  • Rejection of n > 1
  • Session header passthrough
  • Target header passthrough
  • Reasoning level header passthrough

7.2 Integration Tests

Structured output end-to-end:

  • Submit a query with response_format, verify response content is valid JSON matching the schema.
  • Submit a streaming query with response_format, verify accumulated stream is valid JSON.
  • Verify that the agent tool loop runs normally (tools are called) and only the final output is structured.

Evidence end-to-end:

  • Submit a query with include_evidence=true to a workspace with existing conclusions and messages.
  • Verify that evidence.conclusions contains the conclusion IDs that appear in the response.
  • Verify that evidence.messages contains message IDs from sessions that were searched.
  • Verify that evidence.tool_calls contains the expected tool names.
  • Submit a streaming query with include_evidence=true, verify evidence appears in the final SSE event.

OpenAI client compatibility:

  • Use openai Python client to make a non-streaming request, verify response parses correctly.
  • Use openai Python client to make a streaming request, verify all chunks parse correctly.
  • Use openai Python client with structured output (response_format), verify response parses correctly.

7.3 SDK Tests

  • Python SDK: peer.chat() with Pydantic response_format returns parsed model instance.
  • Python SDK: peer.chat() with dict response_format returns JSON string.
  • Python SDK: peer.chat() with include_evidence=True returns ChatResponse with evidence.
  • Python SDK: peer.chat_stream() with include_evidence=True populates .evidence after stream completion.
  • TypeScript SDK: peer.chat() with Zod schema returns typed parsed object.
  • TypeScript SDK: peer.chat() with includeEvidence: true returns typed evidence.

8. Open Questions

8.1 Should response_format support $ref / $defs?

Current answer: No. Requiring inlined schemas simplifies the server implementation and avoids the complexity of JSON Schema reference resolution. Callers can inline their schemas client-side. The Python SDK’s model_json_schema() produces self-contained schemas by default.

Revisit if: Callers frequently need large schemas where $ref would significantly reduce payload size.

8.2 Should evidence include the full observation content or just IDs?

Current answer: Full content. Per the requirements, full observation objects are included (content, level, timestamps, source_ids). This makes evidence self-contained — the caller does not need to make additional API calls to resolve IDs. The size impact is acceptable (see 6.2).

Trade-off: If evidence payloads become too large in practice, we could add an evidence_detail parameter ("full" | "ids_only") to control verbosity.

8.3 Should the OpenAI-compatible endpoint support multi-turn context from messages?

Current answer: No. Only the last user message is used as the query. The Dialectic has its own context system (session history, memory, tool-gathered context). Injecting arbitrary chat history would conflict with the agent’s system prompt and potentially degrade response quality.

Revisit if: Users need to pass application-level context that is not in Honcho’s memory. A future enhancement could inject prior messages as additional context in the user prompt.

8.4 Should evidence include the reasoning trace ID?

Current answer: Placeholder field, populated later. The reasoning_trace_id field is included in the Evidence schema but set to null until the reasoning traces spec is implemented. When traces are stored, the Dialectic can link its trace to the evidence.

8.5 How should the chat() SDK method handle the include_evidence return type change?

Current answer: Overloaded return type. When include_evidence=False (default), return str | None as today. When include_evidence=True, return ChatResponse with .content and .evidence. This uses @overload in Python and TypeScript function overloads for precise typing.

Alternative considered: Always return ChatResponse and make evidence always None unless requested. Rejected because it breaks backward compatibility (callers expect str | None).

8.6 Should the OpenAI-compatible endpoint include evidence?

Current answer: No. Evidence is a Honcho-specific concept that does not map to the OpenAI response format. Callers who need evidence should use the native Honcho API. The OpenAI-compatible endpoint is intentionally a simplified interface for framework compatibility.

8.7 Token usage in the OpenAI-compatible endpoint?

Current answer: Zeroed out. Honcho’s internal token accounting is complex (multiple LLM calls across tool iterations, cached tokens, thinking tokens). Surfacing partial or misleading numbers would cause confusion. The usage field is zeroed to signal that it is not available. This matches the behavior of many OpenAI-compatible proxies.

Revisit if: Callers need usage data for cost tracking. At that point, sum the total input/output tokens across all iterations and report them, with documentation noting this includes internal tool loop tokens.

Graph View

Backlinks