The Summary Is Not the Source

When you deploy an AI summary layer over your vendor risk reports, contract archive, or internal knowledge base, you've quietly changed what your team reads. They're not reading the source anymore. They're reading what the model decided to surface. That's a governance decision, and most teams made it without knowing they made one.

The cleanest illustration this week comes from a Guardian investigation into Tripadvisor's AI review summaries, which glossed over allegations of sexual harassment and described a hotel being sued over hygiene as 'spotless.'

The important part isn't that the model lied. It didn't hallucinate a fact. It selectively omitted the negative ones, and the output still looked accurate and complete. That's the failure mode that scares me, because it passes every eyeball test you'd normally run.

The output looks accurate and complete, and it's systematically biased toward the pleasant reading.

Now, I'd be careful generalizing one platform's feature to every tool. Tripadvisor has an obvious incentive to keep summaries flattering, and your internal knowledge system doesn't sell bookings. But the incentive doesn't have to be commercial for the bias to exist. Summarization models trend toward the smooth, consensus, positive-framed reading because that's what "helpful and concise" optimizes into. The structural pull is the same. The consumer context just made it visible.

In the enterprise it stays invisible until something goes wrong. A due-diligence summary that quietly rounds off the litigation history reads fine right up until the deal closes.

So the question I'd put to anyone running a summarization layer over anything that carries risk: what happens to the bad news? Not "is it accurate on average," but "does it faithfully surface the complaint, the exception, the lawsuit, the one angry reviewer who was right?" If you can't answer that, you haven't automated reading. You've automated the part where someone tells you it's fine.