Ignore the Benchmarks: The Real-World Friction Points of Claude, ChatGPT, and Gemini

---

Benchmarks measure what a model can do on its best day, in a clean room, with the question phrased exactly the way it likes. Real life is a Tuesday. And on a Tuesday, with a deadline, a hangover, and a PDF that’s been scanned sideways, all three of the big models reveal a side of themselves the leaderboards never show. I’ve spent the last several months living inside these tools writing, coding, analyzing, arguing and I can tell you exactly where each one trips over its own shoelaces.

Because here’s the thing nobody puts in the marketing: the capability gap between these models is tiny compared to the friction gap. A model that’s 3% better on a benchmark can feel 30% worse in your workflow if it keeps getting in its own way.

Claude: The Brilliant Kid Who Keeps Hitting the Wall

Claude produces the best prose and the most thoughtful reasoning of the bunch. I’ll die on that hill. But it has a mean streak that shows up at the worst possible moment: the wall. Hit the usage limit in the middle of a long session not the start, the middle and the flow dies. You’re three revisions into something important and suddenly it’s refusing politely and asking you to come back in a few hours, like a library that closes while you’re in the middle of a chapter.

The other Claude quirk is the long-context fade. It will happily read a 100-page document, and then, forty minutes later, it will forget something from page three that’s central to the whole project. I’ve watched it contradict its own earlier answer in the same conversation, with total confidence, like a witness who’s decided the truth is negotiable. When I pushed it past its limits on a money-making experiment, the pattern was unmistakable: brilliant at the start, stubbornly forgetful at the end.

ChatGPT: The Analyst Who Can’t Finish the Handoff

ChatGPT is the most capable all-rounder, and the analysis is genuinely excellent — the code interpreter is still the best tool for real data work in any of these subscriptions. But watch what happens at the handoff, because that’s where the wheels come off. It will produce a perfect cleaned dataset, a beautiful chart, a flawless summary… and then fail to save the file. Or attach the wrong one. Or tell you “done!” and leave the export sitting in a thread you can’t find.

There’s also the instruction drift. You set up a detailed system prompt, it follows it beautifully for an hour, and then suddenly it’s answering like a completely different model that never met your rules. Ask it for the same thing twice in one session and you’ll sometimes get two different formats, two different tones, two different levels of effort. It’s like hiring a brilliant analyst who periodically forgets which job they’re supposed to be doing.

Gemini: The Creative Genius With No Manners

Gemini is the one that surprises people. The creative thinking is legitimately the best — more original, more willing to be weird, better at jumping out of a box it was never in. But the friction is social, and it’s constant. Gemini is the only one of the three that will answer a polite question by sounding mildly annoyed, and it has a habit of rewriting things you didn’t ask it to rewrite. You ask for a single line fix and it hands you back a different paragraph, “improved,” with zero explanation.

Its memory is also the most unpredictable. Some weeks it remembers everything about you; some weeks it greets you like a stranger who’s read your diary and decided not to bring it up. That inconsistency is worse than no memory at all, because you can’t plan around it. It’s the friend who’s brilliant in group chat and never replies to your DMs.

The Friction We All Share

Here’s the uncomfortable part: most of the annoying things aren’t brand-specific. They’re industry-wide, and they’re worth naming out loud.

The apology loop. Every one of these models, at some point, apologizes for something it didn’t do, in a tone that suggests it’s been trained on customer service transcripts and learned the wrong lesson. The confidently wrong answer. The re-doing of work because you trusted a summary instead of checking the source. The subscription tax paying for three tools and actually leaning on maybe one and a half. And the version churn: the model you loved last month gets quietly replaced, and suddenly your careful workflows need re-tuning for a personality you didn’t vote for.

The honest fix isn’t picking the “best” model, because that doesn’t exist. It’s picking the one whose friction you can live with, and building your workflow around its weak spots. The no-nonsense framework I use to evaluate AI tools is built almost entirely around friction, not benchmarks — because benchmarks never made anyone lose an afternoon.

So if you’re choosing between them, ignore the leaderboards. Ask different questions instead. Can you hit the limit mid-task and survive? Can the model finish the handoff, or do you babysit the export? Does it rewrite things you told it to leave alone? Those are the questions that predict whether you’ll still be using the subscription in six months. Because the benchmark says what the model can do. The friction says what you’ll actually get done.