Agents usually remember tool results as blobs of text, then search those blobs by similarity. That’s fine when the question sounds like the original record. It breaks the moment the agent asks anything operational: how many, which came first, what’s missing, what links to what. In other words, most questions an agent asks about its own work. Tool output is already structured. Memory should preserve that structure — not flatten it and hope embeddings can reconstruct the answer later. The result is more accurate answers, fewer tokens, and less guesswork.
Agents remember in the wrong shape
An agent calls a CRM, a CI system, an issue tracker. Each returns JSON with named, typed fields. The usual memory layer throws that away: it flattens the payload to text, cuts it into chunks and embeds them.
Later the agent asks which three builds were slowest, or how many deals closed last quarter. A similarity index can return records that look relevant. It can’t sort them, count them, or show what’s missing.
The structure was there when the data arrived. Memory is where it gets lost.
It gets worse when an answer spans more than one tool. Take an incident report that names a service, a service catalogue that names the owning team, and an on-call rota that names who was holding the pager that night. Nobody declared a key between them. The only link is a value that happens to appear in both: catalog in one payload, svc:catalog in the next.
Similarity search can’t join. It can put two chunks side by side because they read alike, and hope the model links them correctly. When there are two services called catalog, or a team renamed halfway through the quarter, it links the wrong ones and gives no sign it has.
Three chunks that read alike. The model links them itself and answers platform, the owner of the wrong catalog.
SELECT s.owner, r.engineer FROM incidents i JOIN services s ON s.id = 'svc:' || i.service JOIN oncall r ON r.team = s.owner AND i.started_at BETWEEN r.starts_at AND r.ends_at WHERE i.id = 'INC-01'
Each step follows a value from one table to the next. The answer is payments, mensah, or no rows at all.
So the agent pulls in more chunks and makes more calls to compensate, and still ends up guessing.
Keep the structure, and ask it directly
If tool output is stored as typed tables, and the model can use SQL alongside similarity search, it will answer more questions correctly while reading less.
A vector store matching Ingot on structured questions, or Ingot falling behind on semantic ones. Either result means the structure isn’t worth what it costs.
What the benchmark shows
33 questions, seven columns, one model (gpt-5-mini), the same budget of 12 tool calls for each, and 3 runs per question. An answer counts only if it’s exactly right.
H1 and H2 hold: the gap is widest on ordering, where vector scores 33%, and ingot (via API) reads 12.2× less context. H3 does not hold on this run. On semantic questions ingot (via API) scores 89% against 100%, which is the result the hypothesis said would count against it.
Store it the way it arrived
Most of what an agent needs to remember came from a tool, and tools (normally) return structured data. Keeping that structure means you or an llm needs to think about a schema when writing the data. In return, the model gets exact answers and reads far less.
Similarity search still matters, but it should work alongside SQL rather than replace it. That’s what Ingot is.
- That the result holds on corpora a thousand times larger than this one.
- That it holds when payloads change shape between pages.
- What the column mapping costs to write, which this benchmark does not score.