System Map

A walkthrough of how the pieces of Plastic Labs engineering fit together. Read this before working across project boundaries.

The big picture

We ship one product — Honcho — in two ways: as open source for self-hosters, and as a managed SaaS where each customer gets a dedicated Honcho instance. Around that core we run an ML pipeline that benchmarks, generates training data for, fine-tunes, and serves the models that Honcho calls.

graph TB
    subgraph Product
        Honcho["Honcho<br/>memory infrastructure"]
        Kyogre["Kyogre<br/>conversational app"]
    end

    subgraph SaaS["Managed SaaS"]
        Groudon["Groudon<br/>tenant orchestration"]
        Xatu["Xatu<br/>billing pipeline"]
    end

    subgraph ML["ML pipeline"]
        Excadrill["Excadrill<br/>benchmarks"]
        Minccino["Minccino<br/>data trace gen"]
        Machamp["Machamp<br/>fine-tuning"]
        Metagross["Metagross<br/>vLLM serving"]
    end

    Kyogre -->|consumes| Honcho
    Groudon -->|provisions & routes to| Honcho
    Honcho -->|cloudevents| Xatu
    Xatu -->|usage| Stripe[(Stripe)]

    Excadrill -->|boots harness| Honcho
    Excadrill -->|reasoning traces| Minccino
    Minccino -->|SFT/DPO/GRPO datasets| Machamp
    Machamp -->|model weights| Metagross
    Metagross -->|inference| Honcho

Product layer

Honcho — memory primitive

Open-source. Python/FastAPI, PostgreSQL+pgvector, Redis. Primitives are Workspace > Peer > Session > Message. Three background agents form and consolidate memory: Deriver (extract observations), Dialectic (recall), Dreamer (consolidate). API at /v3/{resource}/{id}/{action}. The conceptual core is the Peer Paradigm — both humans and AI are “peers.”

Kyogre — “My Honcho” reference app

React 19 + Vite + Tailwind 4 frontend on Cloudflare Pages; Elysia + Eden Treaty backend on Cloudflare Workers. Supabase Auth, R2 for storage, Stripe + x402 for billing/microtransactions. Demonstrates how an end-user app integrates Honcho sessions, identities, themes, artifacts, and a marketplace. See Kyogre.

Managed SaaS layer

Groudon — tenant orchestration

One tenant = one dedicated Honcho instance on GKE (its own namespace, app-api + app-deriver, its own AlloyDB database). Groudon does not create those Kubernetes resources. It writes rows; the GitOps pipeline renders Helm values and ArgoCD applies them.

Service	Port	Role
Gateway	8000	Public API — resolves the API key to a cluster, proxies to that cluster’s edge-proxy
Admin	8001	Internal tenant/user management
Health	9090	Repair, upgrades, hygiene, placeholder pool
Dashboard	3000	Next.js 15 tenant self-service UI (Vercel)

Gateway, Admin, and Health share a Python/FastAPI core. Control-plane state lives in AlloyDB (groudon schema). Dashboard auth is Supabase (JWT, MFA). Stripe handles billing. See architecture, costs, runbooks.

Xatu — billing pipeline

A separate FastAPI service on the operations cluster (code lives in repos/groudon/xatu/). Receives CloudEvents from Honcho instances → dedupes via Redis → publishes to Redpanda → consumer archives to S3 as Parquet and reports usage to Stripe.

ML pipeline layer

The flow is benchmark → trace → train → serve, with feedback from Metagross’s served models back into Honcho production traffic and back into Excadrill for re-evaluation.

Excadrill — registry-driven benchmark CLI

Boots Honcho harnesses (Docker-based) and runs benchmarks as subprocesses. System benchmarks: BEAM, LongMemEval, LoCoMo, EvolMem, MemoryBench, OOLONG, PersonaBench, TReMu, MemoryAgentBench. Plus 5 direct-context baselines and 3 trace-dependent auxiliaries (Molecular, Coverage, Deriver Quality). Every run produces an immutable sealed job directory with metadata, config, results, traces, and logs. See Excadrill.

Minccino — data trace generation

Takes reasoning traces (live from a HonchoHarness, or replayed from a job dir) and runs them through composable Hydra-configured stage pipelines: source → extract → filter → annotate → transform → synthesize → mix → split → validate → sink. Output is SFT/DPO/GRPO-ready datasets. Adding a new stage is a two-file operation. See Minccino.

Machamp — fine-tuning framework

Hydra-configured SFT and DPO pipelines on unsloth.FastModel (Gemma, Qwen, Llama, Mistral, Phi). Aim for experiment tracking; pluggable artifact sinks for GCS and Hugging Face Hub. Strict rule: no Hydra CLI overrides — every config change goes through editing yaml under conf/. See Machamp.

Metagross — vLLM inference for tinybox

Serves trained models with vllm serve. OpenAI-compatible API. Optimized for the Tinybox V1 Green (6×4090) — typical config runs three independent instances (TP=2 each) behind nginx for max throughput. Also supports Modal (serverless) and Kubernetes deployment. See Metagross.

Where things deploy

Component	Deployment
Honcho (managed)	GKE Autopilot — one namespace per tenant, app-api + app-deriver, 46 clusters
Honcho (self-hosted)	Customer choice — Docker Compose for local dev
Groudon	GKE operations cluster
Xatu	GKE operations cluster
Dashboard	Vercel
Kyogre	Cloudflare Workers (backend) + Cloudflare Pages (frontend)
Machamp training	Tinybox / DGX
Machamp artifacts	GCS bucket + HF Hub
Metagross	Tinybox (Docker, multi-instance) — also Modal & K8s available
Excadrill / Minccino	Wherever a HonchoHarness can be booted (typically Tinybox / DGX)

Everything on GKE is deployed by ArgoCD from git. Telemetry, the 10.x IP plan, and hardware live in Infra.

When you’re working across projects

• Honcho changes that affect billing: check that CloudEvent schemas Xatu expects haven’t drifted.
• Honcho changes that affect benchmarks: re-run Excadrill against the new build; trace deltas may invalidate Minccino datasets.
• New benchmark in Excadrill: think about whether Minccino needs a matching source/ stage to consume its traces.
• New training run in Machamp: artifacts need to land somewhere Metagross can pick them up (GCS or HF Hub).
• Honcho image or chart changes: they land through GitOps (honcho_version + argo-source), not a Fly deploy. Nobody kubectl applys.
• Groudon control-plane changes: writes to the groudon schema trigger a render. Think about the placeholder pool, the health loops, and how a version bump interacts with the GitOps pipeline.