Don't make the model do the math

Well-designed agent tools guard against hallucination

In the last post, I showed an agent computing answers that required reasoning over content and structure. The agent proposed an advocacy strategy derived from the combination of semantics and the graph structure. In that post, I claimed that one benefit of this approach is that each of the agent’s claims can be mapped to specific graph edges that can be verified. I mentioned that in an early experiment, my audit caught the agent getting things directionally right but misattributing votes. I think the failure is instructive in agent design.

The failure

I posed the same small business question to the first version of the agent. It produced an answer that was on the right track but blundered into major factual errors. It got the themes right, identifying a family of bills focused on relief and the opposite family that imposed new fees or mandates. It got the roster of sponsors right. But when it transitioned to the meat of the response: the “cross-aisle Democrats worth cultivating”, it named four Democrats as Nay votes on SB 5525.

Unfortuntaely, zero Democrats voted Nay on SB 5525.

The same response also described a “consistent bloc” of 56 legislators voting Nay on “all or nearly all” of four mandate bills. The real all-four bloc is 22 members. The 56 names were all genuine Nay voters somewhere, but the story overstated how much their votes overlapped.

This verges on worst-case scenario. The response looked credible to someone familiar with the domain. There were specific names, specific bills, and specific vote positions. But many of them were false. Without manual validation, this sort of error can easily slip through unnoticed, undermining the entire analysis.

The diagnosis

Of course, we know LLMs can hallucinate. But the claims were very structured, and almost grounded in the facts: real Democrats with crossover reputations, but the wrong bill. I explored a different hypothesis: that the model didn’t invent the votes, but misassigned real ones.

The working set for that response contained other employer-mandate bills, including HB 1213, a paid-leave bill. In the actual record, Springer, Chapman, and Rule all voted Nay on HB 1213. These were real votes, breaking with their caucus. Richards voted Nay on HB 1213 and HB 2020. All the agent’s “fabricated” SB 5525 Nay votes were a true HB 1213 Nay. One bill-label swap between two adjacent lists accounts for the entire failure.

This specific class of error happens a lot. When the context holds many similar tuples, the model will get “mixed up”. The reason it showed up here and not earlier is instructive: previous runs only drilled down to one bill, so the evidence was compact and easy to attribute. This run was the first exploring a broad set of legislation: fourteen bills and five very similar Nay lists. Summarizing these (”who voted Nay on all four?”) required the model to intersect those lists. It got the list of dissenters right, but got the specific votes mixed up. The 56-member bloc overstatement was the same failure in a different way: set intersection done by the model instead of in code.

Tools maketh the agent

Examining the tools I defined, I discovered I’d made the problem worse. My bill_votes tool returned rows like:

Jim Walsh (R-House): Nay — 3rd Reading & Final Passage

There’s no bill ID here. It only appears in the tool-call arguments, one line above forty rows of results. The graph even stores bill_id denormalized on every VOTED edge, but my initial tool call (in the name of brevity), dropped it. So the model was forced to map rows to bills by where they were in the output. This kind of positional binding across adjacent, similar lists is precisely where things go wrong.

Self-describing rows — HB 1213 | Jim Walsh (R-House): Nay — ... — is a one-line fix that tailors the context for the model.

The real fix

The formatting fix helps, but we’re still just treating a symptom. The real insight lines up with what I discussed in the last post:

This is a failure mode of flat RAG, and while context design goes a long way, ultimately we’re fighting an uphill battle against the model.

For claims like these, the right trust boundary is the boundary of the data layer. Move aggregation into tools:

• voting_bloc(bill_ids, vote) - members who voted a given way on all of these bills

• party_crossovers(bill_ids) - members who voted against their party’s majority, per bill

• verify_claims(claims) - check a list of specific (member, bill, vote, motion) tuples before they’re used in a response.

A “find themes across bills” question needs to do these things, and the tools allow the agent to run them as direct operations on the graph. The new agent chooses to use them. The transcript behind the last post shows voting_bloc and party_crossovers are used to explore the cross bill space, and verify_claims was used to check the named votes before the response was generated. When I added tools to run these operations in the data layer, all the resulting claims could be verified

The takeaway

There’s an obvious takeaway here: give your agents verification tools! The less obvious takeaway is a design principle for agentic retrieval: let the model decide what to compute, but let the data layer compute it. The model’s job is planning and interpretation: which bills constitute the concept, what the crossover pattern means. If the model is also doing all the joins, you risk one of the common failure modes of flat RAG.

This sort of structured, interactive data layer makes agents auditable against a class of “credible but incorrect” hallucinations. Your agent is as good as its tools.