Cognitive psychologists have studied framing effects: the way decisions change under different presentations of the same underlying information. I’m not interested in wading into the machine cognition debate, but given the underlying token-sequence architecture of LLMs it seems worth examining whether they show analogous effects. Given the same information structured differently, how does an agent’s output change?
Researchers at McGill1 randomized the order of a hundred hotel listings on a shopping page, and directed agents to book a room thousands of times. They compared the agents’ choices to data from humans. Unsurprisingly, they found that the agents read more of the page than humans did. But while human attention faded as they moved down the list, the agents tended to focus on the top and bottom items. In some cases, it also changed which hotels got booked. I’ve written about the “lost in the middle” effect before, but that was about whether a model could reliably find data buried in the middle. In this piece, we’ll look at how positioning impacts a model’s choices.
Data and methods
The UCI Online Retail dataset contains over two years of transactions for an online retailer from customers across countries. I built two tables from this data: one for the top export markets, and another for the top products. For each entry I computed revenue growth, number of orders, and cancellation rates. I built an eval set of twelve questions based on the data in these tables. Four questions had clear, verifiable answers, and the other eight were judgement calls based on weighing a few factors.
Clear: “Which product brought in the most revenue in 2011?”
Judgement call: “We’re assigning a dedicated account manager to one export market next year. Which market should it be?
Each question was submitted ten times: five with the table sorted by descending revenue, and five with the rows randomly shuffled. We can use these replicates to examine how often the model disagrees between runs. If we see disagreement between sorted runs, that looks like randomness or unreliability. Disagreement between shuffled runs exposes another possibility: that the model is sensitive to order in its context. I tested this on Anthropic’s 5.5 models (Haiku, Sonnet, and Opus).
Clear answers are stable
The good news is that, for questions where the answer was clear, the models returned consistent results whether or not the context was shuffled. When asked “Which export market has the highest average basket?”, the models always returned “Netherlands”. This should bring us some comfort, given the heavy adoption of LLMs for question-answering.
The ambiguity trap
The judgement call questions are where we run into trouble. All three models had higher self-disagreement on these questions. This isn’t a huge surprise given the built-in ambiguity. For example, if asked which product to include as “a free sample in every new-customer welcome pack,” two humans may draw different conclusions. While we might accept this sort of variance, we would want it to be based on different arguments or approaches rather than noise due to data presentation.
In this eval set, Opus and Haiku showed some run-to-run variance given the same context. Their responses also vary when the context is shuffled, but the difference is not statistically significant. Sonnet yields a different story. When Sonnet was presented with a sorted table, it chose the same item (party decorations) every time. But when the table order was shuffled, it switched between that or three other items (a chalkboard, a Christmas decoration, and storage bag). The model returned a reasonable argument for each, based on cost, appeal, and return rates. If you gave this model decision power, how would you know whether to trust the answer?
Problems of the sort
The first objection might be, “I will simply logically sort my tables, instead of randomly shuffling them like an agent of chaos.” It’s a fair point, but (a) we may not always know what sort order is right for the question, and (b) the direction of the sort is a choice itself.
I ran every question with the sort order reversed (ascending). On the “account manager” question above, Opus and Sonnet both chose France when the table was sorted by descending revenue, and both chose Australia when the sort was reversed. Both of these are defensible choices: France is the bigger market by number of customers and shows steady growth. Australia is much smaller, but grew several times on a few large accounts. Both of the sorts are defensible, too. But we get two different answers, and they’re consistent between models. The sort order is an input into the decision, and we might not have chosen it intentionally.
Are your evals missing this?
If you’re running evals (and you should be!) there is a chance that you’re still missing this inconsistency. The reason for this is simple: evals are easier to write on simple questions with clear answers. As we saw in this analysis, those are the types of questions that don’t move with presentation order.
Even if you’re running the judgement calls, if you only run one context through you may only get one answer. Most of the time, evals vary the question with fixed context. In this case, we did the opposite: we used the same question and data with a different arrangement. This is one check you can add to some of your evals. Shuffling context in a few scenarios will give you a measure of consistency.
Takeaways
This is a small sample study, but it’s enough to demonstrate that that the models show something akin to framing bias. When there’s a clear answer, maybe that framing doesn’t matter too much. But when it comes down to judgement, presentation choices impact the models’ decisions in ways you may not intend or anticipate. We saw how two reasonable sorts of the same table change the recommendation.
In Don’t make the model do the math, I wrote about how whenever possible, anything that can be computed in the data layer should be done there and handed to the model. Row order fits there, too. If the question turns on growth, sort by growth. If margins matter, return them. These extra fields can give the model the context it needs to ground a tiebreaker, rather than relying on unstated inputs. Once these choices are in the tool output, they’re choices you can inspect and analyze explicitly.
Ultimately, good judgement means someone has to decide what “good” means. If you can do that, you can build structure under which a model can apply the standard consistently. But without it, you’re relying on the model to set the standard, and it may choose factors you wouldn’t.