Honcho on Neocloud — Reference Architecture

The sober engineering answer to “how would Honcho actually run on our infrastructure?” — which every Tier-A meeting will ask. One reference architecture, two supported deployment topologies, one joint-build lane, and a fenced “we will not say this in a meeting” appendix. Grounded in the current Honcho codebase (Postgres + pgvector + Redis + FastAPI, AGPL-3.0). Companion docs: Distribution Thesis · Landscape Map · Neocloud Lexicon · Vendor Dossiers.

The discipline here is the inverse of the rest of the corpus: the Landscape Map and Lexicon are about opening rooms; this document is about not breaking promises in the room. If something isn’t in the codebase today, it doesn’t appear here — or it appears in the out-of-scope section labeled as such.

What Honcho is, architecturally

Four services:

Service	What it does	Image / process
api	FastAPI HTTP server (port 8000). Message writes, context/representation/search queries.	Python; repo Dockerfile.
deriver	Background worker. Processes the reasoning queue, generates conclusions, updates representations.	python -m src.deriver, same image as api.
database	PostgreSQL + pgvector. Source of truth for messages, peers, sessions, representations, embeddings.	pgvector/pgvector:pg15. Required.
redis	Job/cache plane coordinating api ↔ deriver.	redis:8.2. Required.

Constraints that drive every integration decision:

• Postgres + pgvector is non-negotiable. Connection URI uses the postgresql+psycopg prefix. No MongoDB driver, no MySQL adapter, no DynamoDB connector. Honcho is a Postgres application.
• The LLM provider is configurable via OpenAI-compatible HTTP transport. All reasoning calls (deriver, dialectic, summary, dream) go over a chat-completions interface. Anything OpenAI-shaped works: OpenAI, Anthropic (native), Gemini (native), or any OpenAI-compatible endpoint via a BASE_URL override (Together, Fireworks, vLLM, Ollama, LiteLLM, OpenRouter). This is the single most important commercial fact in this document.
• Reasoning is asynchronous. Writes enqueue background work; the deriver processes it and updates representations. Reads return what’s materialized.
• License: AGPL-3.0. Matters for any topology where a neocloud forks/modifies the code.
• Distribution: managed at api.honcho.dev, or self-host via docker-compose. No first-party Helm chart, Terraform module, or K8s operator today (custom K8s is feasible — they’re standard containers — but not a blessed shipping artifact).

The reasoning-as-inference fact

Honcho’s deriver, dialectic, summary, and dream subsystems all make LLM calls. Every message ingested triggers background reasoning; every peer.chat query at any reasoning level is a model call; every summarization is a model call. Honcho-using customers make materially more inference calls than they would against the model directly — Honcho reasons about the conversation as well as participating in it.

When the question becomes whose model serves those calls, the answer is whoever’s endpoint is in the deployment’s env vars. Pointing DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL (and the dialectic/summary/dream equivalents) at a neocloud’s OpenAI-compatible endpoint routes the entire reasoning workload there. A one-line config change, not a code change. That is the inference pull-through, made concrete.

The two supported topologies

Both work today with no new code from Plastic Labs or the neocloud.

Topology A — Managed Honcho + Neocloud Inference (the default)

Customer uses the managed Honcho service at api.honcho.dev; the neocloud hosts inference for both the customer’s agent and Honcho’s internal reasoning (routed via LLM config on the managed cluster).

• Customer: one env var to point reasoning at the neocloud; no infrastructure to operate; state lives in Plastic Labs’ managed Postgres.
• Neocloud: their inference SKU absorbs both the customer’s calls and Honcho’s reasoning calls (incremental token volume that wouldn’t otherwise exist); zero engineering work; co-sell with referral/SPIFF economics.
• When: the customer wants memory now without running database infrastructure. The default first deployment in every Tier-A pilot.
• Requires: nothing new — no neocloud Postgres, no neocloud integration, no Plastic Labs shipping. The pilot deck can promise this today.

Topology B — Self-Hosted Honcho on Neocloud Compute

Customer runs the four-service stack on the neocloud’s compute (VMs / K8s / containers); reasoning LLM points at the neocloud’s inference endpoint; Postgres + Redis provisioned by customer or neocloud per deal shape.

• Customer: all services and state on the neocloud’s substrate — satisfies data-residency / compliance / sovereignty without giving up Honcho.
• Neocloud: incremental compute (4 containers + Postgres + Redis); inference revenue routes through their endpoint same as Topology A; storage is BYO from Honcho’s perspective (any Postgres with pgvector — their managed Postgres where it exists, otherwise Postgres on their compute).
• When: data-residency/regulatory requirements preclude third-party SaaS memory; deal is large enough that running Honcho infra is a feature, not a tax.
• Footprint: neocloud runs four standard containers + a pgvector Postgres, surfaces /health and Prometheus metrics through existing observability. Plastic Labs provides the docker-compose template, env reference, and runbook — all exist today.

Configuration surface

The whole inference integration is a small set of env vars on the Honcho deployment:

LLM_OPENAI_API_KEY=<customer's neocloud API key>

DERIVER_MODEL_CONFIG__TRANSPORT=openai
DERIVER_MODEL_CONFIG__MODEL=<model on neocloud, e.g. "deepseek-v3.2">
DERIVER_MODEL_CONFIG__OVERRIDES__BASE_URL=https://<neocloud-inference-host>/v1

DIALECTIC_LEVELS__minimal__MODEL_CONFIG__MODEL=<small/fast>
DIALECTIC_LEVELS__low__MODEL_CONFIG__MODEL=<small/fast>
DIALECTIC_LEVELS__medium__MODEL_CONFIG__MODEL=<mid-tier>
DIALECTIC_LEVELS__high__MODEL_CONFIG__MODEL=<mid-tier>
DIALECTIC_LEVELS__max__MODEL_CONFIG__MODEL=<frontier>
SUMMARY_MODEL_CONFIG__MODEL=<small/fast>
DREAM_DEDUCTION_MODEL_CONFIG__MODEL=<mid-tier>
DREAM_INDUCTION_MODEL_CONFIG__MODEL=<mid-tier>

EMBEDDING_MODEL_CONFIG__TRANSPORT=openai
EMBEDDING_MODEL_CONFIG__MODEL=<embedding model>
EMBEDDING_MODEL_CONFIG__OVERRIDES__BASE_URL=https://<neocloud-inference-host>/v1

Model-tier guidance maps onto neocloud SKU tiers: light tier (deriver, summary, dialectic minimal/low) → cheap/high-throughput models; medium tier (dialectic medium/high) → reasoning-capable mid-cost (~$0.30–$1.50 / M output); heavy tier (dream, dialectic max) → frontier class. Per-tier provider routing is independent — a customer can route light reasoning to the cheapest endpoint and heavy to a frontier provider, or everything through the neocloud. We make config trivial; we don’t prescribe.

Where each Tier-A neocloud sits today

The pattern is uniform: Topology A works on all five today, no engineering required. Topology B is mechanically possible everywhere; the storage-layer story differs. None of the five sells a managed Postgres SKU today — in Topology B the customer brings their own Postgres (RDS, Supabase, Neon, or self-hosted on the neocloud’s compute). CoreWeave has an open Managed Databases role; a managed Postgres SKU could land mid-2026 and turn its Topology B from customer-managed to neocloud-managed — worth tracking, not promised.

Joint-build lane (not shipped today, but achievable if a deal asks)

Lets us say “yes, we can do that” without committing it as a current feature. Nothing here goes in the standard deck:

• First-party Helm chart; Terraform / IaC artifacts.
• Neocloud-specific deployment guides (“Deploy Honcho on CoreWeave / Together GPU Clusters”).
• Co-engineered reference architecture with a managed Postgres provider (Neon / Supabase / Aiven). MongoDB as a backend is not this — it requires a code change, not a recipe. Do not promise it.
• Documented recommended-config page per neocloud (their fast/cheap model in catalog).
• Custom-trained reasoning models on a neocloud’s hosted endpoints (Together Model Shaping, Baseten Loops, Fireworks RL) — genuinely speculative; only if the neocloud raises it first.

Explicitly out of scope — do not say in a meeting

• “Honcho can use MongoDB as a backing store.” Not true. Honcho is Postgres + pgvector. The MongoDB story is co-listing / complementary-service, not storage.
• “Honcho is a Chainlet inside Baseten Chains.” Wrong framing — it’s an external HTTP service a Chainlet calls (parallel, not nested), structurally like the Baseten + Atlas reference.
• “Honcho replaces / competes with [neocloud’s product].” Honcho is complementary to inference, sandboxes, catalogs, retrieval, observability, RAG. If a BD lead hears “competing with our roadmap,” the deal is dead.
• “We have a Helm chart / Terraform module / K8s operator.” We don’t — honest answer is “supported via docker-compose; first-party K8s artifacts are on the joint-build lane.”
• “Honcho’s reasoning models are trained on neocloud X’s infra.” Plastic Labs trains them; they run wherever inference routes.
• “Honcho memory and [neocloud RAG/vector DB] compete.” They layer: RAG retrieves documents, Honcho reasons about peers. Both in the same agent.

The one-paragraph pitch a BD lead memorizes

Honcho is a four-service Postgres-backed FastAPI system that runs either on our managed cluster at api.honcho.dev or self-hosted on any container runtime. Customers deploy it managed by us with inference routed to your endpoints (Topology A — the default), or self-hosted on your compute with the same routing (Topology B — for data-residency/sovereignty). The inference workload Honcho generates is incremental to whatever the customer already runs — every memory write triggers background reasoning, every memory query at any reasoning level is a model call. Pointing those calls at your inference SKU is a one-line config change, no code on your side. Storage is a Postgres + pgvector requirement the customer satisfies any way they like; if your platform ships managed Postgres later, that’s a natural co-engineering integration, but it’s not a blocker today.