This week, while auditing the SEO of my own website with an AI assistant, I learned a great lesson. The AI confidently announced: “Your blog only contains 5 articles.” Then it built an entire diagnosis on top of that — “your blog is too thin,” “there are ghost URLs,” “your local code is out of date.”
Except my blog contains 18 articles. All of it was wrong. And the story of why it was wrong is, I think, one of the most useful lessons when working with AI.
To count the articles, the AI used grep — the classic command-line text-search tool. Something like “count the lines containing slug:.” grep's answer: 5.
The problem: my articles are stored in the code with all their HTML content on a single line — sometimes 20,000 characters long. And there, the grep shipped by default on macOS (BSD grep) has a known flaw: on extremely long lines, it skips matches or gives up — without ever showing an error.
In other words: the tool wasn't built for this. grep is designed for text with “one piece of info per line.” On structured data (code, JSON) where a single line weighs 20 KB, it leaves its comfort zone and starts miscounting.
Here's the important part, and it goes far beyond my little blog.
The problem isn't that the tool crashed. It's that it lied without saying so. No error, no warning. It returned “5” — a perfectly believable number. And an AI, like a person in a hurry, trusts a believable number.
That's the number-one trap when building with AI:
A loud failure, you see. A silent failure, you believe.
An AI is never more reliable than the tools it uses. If the tool returns data that's wrong but plausible, the AI will build an entire line of reasoning — confident, articulate, convincing — on sand. And the more eloquent it is, the more dangerous: you don't feel like double-checking an answer that “sounds right.”
What saved the diagnosis was a simple reflex: when a result surprises you, cross-check it with another method. Instead of grep, we loaded the file with a real parser (a bit of Node code that actually reads the data structure and counts the items). Answer: 18. Instantly, the whole shaky diagnosis collapsed, and the real one appeared.
The three rules I take away, and apply in every system I build:
Counting objects in code or JSON with a text search is fragile. A parser (Node, jq, Python) reads the structure, not the characters. It gives the right answer, or a real error — never a silent “roughly.”
If you really must search text, use a robust tool (ripgrep, GNU grep) rather than the default macOS grep, which stumbles on long lines.
“Half your blog disappeared” should have triggered doubt, not an action plan. Doubt at the right moment beats a thousand corrections after the fact.
Because this is exactly what separates a gimmick AI from an AI system you can actually run a business on.
Deploying AI in a company isn't plugging in a model and believing its answers. It's building guardrails around it: verifying outputs, cross-checking critical data, choosing tools that fail loudly rather than silently. An AI that automates your quotes or your follow-ups has to be reliable when no one is watching — and that is designed, not improvised.
I got fooled for five minutes by a 50-year-old tool. The difference isn't never being wrong — it's having the reflex to verify, and building systems that make errors visible.
Want to bring AI into your business — but properly, with guardrails? That's my whole job. Let's book a free one-hour audit: we'll look at what, in your case, can be automated reliably.
Benjamin Bertigny — Founder of NBN IA. I build custom AI systems for SMBs. Anthropic-certified, trained at Alegria Academy. Based in Bordeaux, serving all of France.
Ready to automate your business?
Free 1-hour audit — we identify your 3 most valuable automation opportunities together.
Book my free call