Plastic Labs Internal Wiki

Recently Updated

Architecture Decision Records

12 min read

The purpose of this page is to document different architectural decisions made during the development cycle of Honcho. We will append to top to have the most recent changes most accessible.

Architecture Decision Records

Peer Paradigm [06.17.2025]

Below are a collection of design decisions that came about when moving Honcho from the original assistants api paradigm to the peer paradigm.

Generally speaking what we found in our design meeting was that Honcho in it’s current form was trying to do too much to point where it was acting like an entire database. By exposing metamessages we were encouraging people to encode their cognitive architecture into honcho, but honcho was not designed for complex queries. Similarly exposing collections presented another anti-pattern. We are never going to be the best database or vector database and we shouldn’t encourage people to use Honcho like that.

The original argument was that provisioning a database is hard for a vibe coder, but alternatively the language model will know how to setup and provision a database really quickly, especially something like sqlite. Instead we should focus on just what is the core value proposition of Honcho — the Dialetic Endpoint. The assistants paradigm was also restricting because we were entering territory where we want to have multiple users or agents all interacting with each other and have social cognition on each other. So we decided to cut metamessages and make collections and internal feature only so the dialectic can access it, but get developers to lean on their own vector dbs. Honcho should be focused on identity modeling and giving social cognition, not being a all encompassing database.

Group chats needed to be a native aspect of Honcho to be more future proof. Towards that end we decided to switch to Peer which is an abstract entity that could be a human, agent, NPC, etc.

Sessions can have multiple Peers with every Peer able to form a representation of all the others. This also makes it a bit more native to Protocol version of Honcho and provides a cleaner path towards migration.

A unique feature of the Peer paradigm is the ability to have local and global representations of Peers. Each Peer has it’s own representation of other Peers based on messages it has seen that Peer send, but there is a global representation of a Peer owned by itself based on everything it has sent. This is useful depending on what kind of application you want to build.

  • In simple chat apps like Bloom you probably only care about the global representation
  • If you’re building a game with NPCs you’d want your agents to be able to form their own representations of the other Peer’s based on their experiences.

Another thing we are exposing is the ability to ingest arbitrary data into a Peer by adding messages to it directly. This is useful for updating the global representation of the user.

Below are a series of different implementation details and decisions we made while designing the Peer Paradigm. Part of this work also has to do with the RSR work done to update the method in which user representations are updated.

erDiagram
    Workspace ||--o{ Peer : "contains"
    Workspace ||--o{ Session : "contains"
    Peer ||--o{ Session : "participates_in"
    Session ||--o{ Peer : "has_participants"
    Peer ||--|| GlobalRepresentation : "owns"
    Peer ||--o{ LocalRepresentation : "owns"
    LocalRepresentation }o--|| Peer : "represents"
    GlobalRepresentation ||--|| Collection : "stored_as"
    LocalRepresentation ||--|| Collection : "stored_as"

    Workspace {
        string id PK
        string name
        jsonb metadata
        datetime created_at
        datetime updated_at
    }

    Peer {
        string id PK
        string workspace_id FK
        string name
        jsonb metadata
        datetime created_at
        datetime updated_at
    }

    Session {
        string id PK
        string workspace_id FK
        jsonb metadata
        datetime created_at
        datetime updated_at
        boolean is_active
    }

    GlobalRepresentation {
        string id PK
        string peer_id FK "owner"
        string collection_id FK
        datetime created_at
        datetime updated_at
    }

    LocalRepresentation {
        string id PK
        string owner_peer_id FK "who owns this representation"
        string target_peer_id FK "who is being represented"
        string collection_id FK
        datetime created_at
        datetime updated_at
    }

    Collection {
        string id PK
        string name
        jsonb metadata
        datetime created_at
        datetime updated_at
    }

✅ Can you scope a dialectic query to a session?

Conclusion: No

The terminology has changed where dialectic queries only refers to a Peer level query where there is a query against the global set of facts and a synthesis prompt that will put it all together to come up with an in context representation.

At the session level we instead want to just expose the working representation that the deriver uses for processing messages in a session. This should just be a get_representation endpoint that returns the working memory

It is up to the application to decide if they need more context after that.

We should still allow develoeprs to pass in an optional session_id to the Peer.chat function so the dialectic can pull in additional context.

✅ Where should working representations be stored?

Conclusion: session_peers table

The deriver is keeping a JSON document with various facts that are relevant and useful in the context of a session and currently those are being stored in a metamessage. We are only ever using the latest one for each successive message we derive from and thus we don’t need to store every previous one. The tricky part is that now sessions have multiple peers that are all kicking off various deriver processes and each Peer will have it’s own representation. So we can’t store that working memory in the metadata of the Session and Metamessages no longer exist. So instead we can keep it in the session_peers table where the working representation for a peer in a session is stored in the row that specifies that a peer is in a session

What should the endpoint / SDK logic look for getting a working representation?

An example of a working representation / working memory

"final_observations": {
    "explicit": [
      {
        "content": "User said: 'Hey Mel!' - addressing someone named Mel",
        "created_at": "2023-05-08T13:56:00+00:00"
      },
      {
        "content": "User said: 'Good to see you!' - expressing positive sentiment about seeing this person",
        "created_at": "2023-05-08T13:56:00+00:00"
      },
      {
        "content": "User said: 'How have you been?' - asking about the other person's recent state or experiences",
        "created_at": "2023-05-08T13:56:00+00:00"
      }
    ],
    "deductive": [
      {
        "conclusion": "The user believes they have encountered 'Mel' before",
        "premises": [
          "User said 'Good to see you!' which implies previous encounters"
        ],
        "created_at": "2023-05-08T13:56:00+00:00"
      },
      {
        "conclusion": "The user is initiating a conversational exchange",
        "premises": [
          "User said 'Hey Mel!' as a greeting",
          "User asked 'How have you been?' which is a conversation starter"
        ],
        "created_at": "2023-05-08T13:56:00+00:00"
      }
    ],
    "inductive": [],
    "abductive": [
      {
        "conclusion": "The user believes they have an established relationship or familiarity with 'Mel'",
        "premises": [
          "Casual greeting format 'Hey Mel!'",
          "Familiar expression 'Good to see you!'",
          "Personal inquiry about wellbeing"
        ],
        "created_at": "2023-05-08T13:56:00+00:00"
      }
    ]
  },

✅ Should we expose semantic search to the developer?

Conclusion: Yes semantic search across history is very helpful

This question is nuisanced because you could expand it into what should we expost to the developer. There is both:

  • The embedded message history
  • The representations that exist as embedded derived facts

The thought process that we came to was that for accessing representations you should really always leverage the Dialectic Endpoint, but we could see how it is useful have access to semantic search of the message history.

We’ll expose the search against the message history via .search() methods. These will be scoped at different levels

honcho.search() # workspace level
session.search()
peer.search()

Now embedding every single message could get quite expensive quite fast and might not make sense in all deployment scenarios. So we should make it a feature flag on the infrastructure level. Meaning we can gate it behind an environment variable or config value.

EMBED_MESSAGES=true|false

The main motivation for embedding messages was actually to make it accessible to the Dialectic Endpoint so it could agentically traverse the messages and build up it’s context, but makes sense to expose that specific feature to the user if possible.

In our managed instance of Honcho we’ll expose this.

✅ Should we expose full text search to the developer?

Conclusion: Yes

In the same way that we want to embed all of the messages we wanted to add a full text search index to give the Dialectic Endpoint the ability to agentically build up it’s context. We should similarly expose this, but abstract it away from the user with the .search() method just doing whatever is enabled.

This should also be a feature flag on the infrastructure level.

FULL_TEXT_SEARCH=true|false

✅ What should the Queue look like now that we have various different deriver tasks

The Queue changes dramatically in this paradigm as it can potentially balloon in scope. Instead of just enqueueing one task to on every sent message there is an arbitrary number of tasks that need to be scheduled.

For every message there’s:

  • Updating the global representation of the sender
  • Updating the local representation of the sender for every other Peer in the session
  • Summarizing the session if we’ve hit a threshold

How could this payload look like:

{
  workspace_id: <>,
  sender_id: <peer_id>,
  target_id: <peer_id>,
  message_id: <>,
  task_type: "summary|representation"
}

We would still have a column for session_id as there is still the requirement of messages in a session need to be processed in order

However, because there are different representations being processed we can still parallelize those updates even if it’s the same session. So the ActiveQueueSessions table can change to look like:

session_id, sender_id, target_id, task_type

We decided that we can still just use one table for this rather than having totally separate queue tables for each type of task.

There’s also the question of should the scheduler be greedy or not, meaning should it schedule every single potential task or check if tasks are valid.

Aside #1 Feature Flags

Before talking about the scheduler another thing we need to think about is feature flags you can set on Peers and Sessions that effect what work the deriver needs to do.

There are many scenarios where you may not want any local representations or do not care about modeling a specific Peer at all. In those cases you need ways to modify the behavior of the scheduler. The 3 flags we came up with are:

session_peer flags:

  • Should a Peer observe other Peers (default: False)
  • Should others observe this Peer (default : True)

Peer level flags:

  • Should this Peer be observed at the global level (default: True)

This makes it so local level representations are turned off by default we only care about global ones. It also let’s us create complex logic such as having a group chat where two human users want to model each other but ignore a support bot.

So now going back the scheduler, should it enqueue every potential task onto the queue or should it check if the task is valid before enqueuing it.

The queue should definitely have different items for each task so that the deriver workers can parallelize more efficiently.

When a new message is sent the scheduler will at the very least need to make a query against the session_peers table to get all of the peers in the session. To get the additional context it needs on the flags it would need to do a joined query on the Peers table. The argument for not doing the check initially is that there’s less latency on the a smaller query and it is less likely to fail.

The deriver workers are much more resilient and able to recover from arbitrary errors, while the background task scheduler is not. However, since there is still a required query, there is not a significant reliablity difference in not doing the join. This also prevents us from adding too much noise to the queue that slows down the workers.

✅ Should there be a limit on the number of peers in a session

Conclusion: Yes

There should absolutely be some way of setting a limit as it can exponentially grow the size of the queue based on the number of peers in a session. We can make this a infrastructure level feature flag with an environment variable to set the MAX Peers in a session. The default should be 10.

MAX_PEERS_IN_SESSION=10

✅ How can you poll the status of a queue

We definitely need to expose the ability to poll the status of the queue to understand when the representation is fully up to date.

In the same way we are making everything Peer centric, we can make the status checks on the Peer

peer.rep_status(target: Optional[Peer|peer_id])

if nothing is specified it will check it’s global representation queue, otherwise look at the local representation queue

Should be pending work with a number of remaining items

{
  "status": "pending",
  "remaining": 3
}

✅ What is the relationship of the working memory and the different types of derivers?

There are local level derivers and global level ones that are creating representations of the Peers. We can probably just store all of these in the metadata with IDs for who the representation is in relation to

the dialectic calls will be restrict to the Peer level while the working representation should be used to get a quick set of data to inject into a prompt. We would expose it via an endpoint get_representation

peer.session.get_representation → working representation

peer.chat → dialectic

✅ How do you modify feature flags on a session_peer level

Mentioned a lot of the details of the feature flags in the aside above. This section is interested more in the ergonomics of the SDK for adjusting the feature flags.

You should be able to set the flags when you add Peers to a session using the fallback, default values.

session.add_peers([...(peer_id, flags)], default_flags)

session.get_peers() should return the Peer objects themselves, rather than the settings of the Peer. We don’t want to introduce a new type for just the Peer in a session. It also should not be a super useful pattern of wanting to get all of the configs at once. So we can expose it via getter and setter methods.

session.get_peer_config(peer_id)
session.set_peer_config(peer_id, config=config)

Graph View

Backlinks