Cleo remembers. Past conversations, decisions a business owner has made, documents they have written with her. On each turn, a retrieval step searches that memory for material relevant to what the user has just said, and the results are placed in front of the model.
The search worked the way retrieval-augmented systems usually do. Embed the user's message as a vector, find the stored material whose vectors sit nearest, include the top results. Messages under ten characters were skipped, since there is little to search on.
Consider what that does with an ordinary reply. "Yes, go with the second one." That is more than ten characters. It was embedded and searched, literally. Whatever in memory sat closest to the phrase "yes, go with the second one" came back, presented as relevant.
Measured, not imagined
This was not a hypothetical. A nightly check scores a sample of retrievals for relevance, and it had been showing that around one in six recalled conversation snippets, and nearly a third of retrieved document passages, had nothing to do with the turn they were retrieved for. A large share of those came from turns like this one: a nod, a choice, a short follow-up, where the words carry almost no topic and the topic lives in the turn before.
Search on the subject
The fix is a small classifier that runs before retrieval. It decides whether a message is a substantive question or statement, or a nod or follow-up that depends on what came before. A real question is searched exactly as typed. A nod or follow-up is combined with the user's previous substantive message, and that combination is what gets searched.
It is a vocabulary test and a short list of patterns. No model call, no network, no added latency. I considered asking a small model to rewrite the query, which is the more common approach, and decided against it for now. Every turn would pay for it, and the deterministic version could be tested exhaustively.
The ten-character skip is unchanged, so no turn that previously retrieved nothing starts retrieving now.
Corrections look like follow-ups
The hard part was not recognising "yes". It was recognising corrections.
"We tried that one already." "I said no to that one." These share most of their words with follow-ups. Combine them with the previous turn and you retrieve, with great confidence, the very thing the user is rejecting. The filter for this kind of narrative is the part of the classifier that carries the weight. Nine lookalike phrases are tested in both directions, and the request forms of the same verbs, "try that one", are asserted not to trigger it.
Smaller fixes alongside
Recalled snippets now carry the title and date of the conversation they came from, fetched in one batched query rather than one per snippet. A memory with a date is easier for the model to weigh than one without.
Each turn's trace now records the query that was actually searched, not the raw message, so the nightly check scores the pair that really existed. Otherwise it would be marking a reshaped query against the original words and calling the result a miss.
And on a turn where retrieval is skipped, the context no longer says the search came back empty. No search was made. Saying it returned nothing invites the model to conclude there is nothing to remember, which is a different and incorrect statement.
The general point
A retrieval query is not the same thing as a user's message. People speak in conversation, which is full of reference to what has just been said. Search, vector search included, has no conversation. Somewhere between the two, something has to decide what the turn is about. If nothing does, the search will faithfully find whatever sits nearest to the word yes.