this aspect of honcho is primarily concerned with predicting user mental state given any arbitrary data
Post Raise Summer ‘24
i chatted with claude about an approach where we generate samples of thoughts (mental state imputations) per user message and fine tune a model based on which thoughts best predicted responses in the bloom dataset (or any dataset) based on an LLM as a judge. could probably just manually use a human as a judge, don’t need a ton of examples.
so using the emergent ability to improve the explicit ability
reason i say probably can do this manually is because of the success andy ayrey had with truth_terminal using only ~500 examples of his backrooms runs.
this is also why i want to invest in my personal assistant to collect this training data for myself. could be a good demo / PR campaign
Historical Recap
as of 2024-07-15, the best method is and has been up and to this point violation of expectation (implemented here). the mental state imputation aspect of this framework is done entirely by prompting existing language models. eliciting emergent ability