Ayush has been working on this during summer of ‘24. We wanted to take a stab at building a framework that allowed us to test the 4 buckets more effectively, as opposed to popular ToM benchmarks that test ToM in ways we think fall short.
Some features of this framework we think are important:
- multi-turn conversations
- interchangeable models, prompts
- flexible user representations
Humans get to know each other by spending time talking to each other. Because it takes time, measuring true ToM requires a framework that allows us to simulate time spent, conversations had, mental models updated, etc.
Ayush is using a relatively simple experiment to build and test this framework
Starting w/ a user representation?
It has become clear that whether or not we have a user representation to start determines much of the downstream tasks / evals.
For example, in the paper we didn’t assume any user representation existed and relied solely on the LM’s ToM abilities implemented in VoE to construct it. This enables an entirely passive mode of user representation construction, and the eval metric is whether or not that representation aids in predicting ensuing user behavior (ground truth we DO have).
In Ayush’s experiment however, he’s bootstrapping a user representation using a LM and evaluating the re-construction of that representation after a multi-turn conversation. Here, we want to measure our proximity to the original user representation. Predicting user behavior is useful i.r.t. improving the re-construction of that representation.
Courtland summarized it nicely:
synthetic users good for measuring representation delta
human users good for measuring behavior delta
It’s a “result of the nature of those two entities”. If we have a user representation, we have access to that user’s internal mental state and can predict it directly. If we don’t have a user representation, we don’t have access and therefore need to predict user behavior as a proxy to the user’s internal mental state.