Some of Your Best Numbers Might Not Be Real

Most businesses check their numbers when something looks wrong. A campaign underperforms, a target gets missed, a client complains, and someone finally opens the dashboard properly instead of skimming it. Far fewer businesses check their numbers when something looks right. A good result rarely gets audited. It gets celebrated, then used to justify doing more of whatever produced it.

That asymmetry is a real gap, and it's worth being precise about why it matters. A bad number that's wrong at least points you toward checking something. A good number that's wrong points you toward doing more of something that was never actually working, with more confidence than the truth would ever have given you.

A specialist architecture firm's search campaign made this concrete. The account was in trouble on the surface, spend going out, almost no leads coming back, the client ready to pause billing entirely. That part of the diagnosis is its own story: the niche simply didn't generate enough monthly search volume to justify the budget behind it, a demand-side ceiling rather than an execution failure. But while reviewing performance to understand that ceiling, a second, separate issue surfaced. A CAPTCHA-related tracking bug had been inflating the conversion numbers the whole time. Some of the early "wins" the client had been counting weren't real conversions at all. They were ghost data, an artefact of a broken form, being read as evidence the channel was working.

Put the two findings together and the account had two genuinely different problems at once, not one. A demand ceiling that explained why volume was low, and a data integrity problem that had been quietly overstating what little volume there was. Fixing only the first without catching the second would have left the client making decisions off a number that flattered a channel it shouldn't have. The record was corrected, even though correcting it meant admitting the channel was performing worse than it appeared, not better. That's a harder conversation to have with a client than "here's a new problem we found." It's "the thing you were pleased about wasn't actually true."

This is the same discipline behind Halo's First Law, diagnosis before prescription, pointed in a direction it doesn't usually get pointed. Diagnosis is normally understood as finding the hidden problem behind a disappointing number. It's less commonly understood as checking whether an encouraging number is actually earned. Both are the same underlying habit: don't act on a number until you've confirmed what it's actually measuring. A separate engagement, a media and sponsorship sales account covered in Accountability Isn't Only About Finding the Hidden Constraint, shows the other side of the same check: sometimes checking a good result confirms it was real. Two million impressions, a 2.5% click-through rate, 80%+ video completion, all held up under scrutiny. That's not a contradiction of the architecture firm's case, it's the same discipline producing a different, equally honest answer. The point was never "assume good numbers are fake." It's "don't assume they're real just because they're good."

The architecture firm case is also a useful reminder that a tracking or data problem isn't always the same kind of problem twice. "We Don't Have the Data" Is a Claim Worth Checking covers a case where the data existed but nobody had translated it into something usable, a translation failure. This is a different failure mode entirely: the data existed, was being read, and was simply wrong, a corruption failure. They can look identical from a leadership dashboard, "we're not sure we can trust this number," and they need different fixes. One needs someone to sit down and read what's already arriving. The other needs someone to check whether the number was ever measuring what it claims to measure in the first place.

A few practical checks worth running before trusting a good number the same way you'd already be inclined to question a bad one:

• Audit your best-performing metric as carefully as your worst one. A campaign, channel, or number that looks unusually good deserves the same scrutiny as one that looks unusually bad, not less, precisely because nobody's instinct is to go looking for a problem behind good news.
• Check what a "conversion" actually requires to fire. A form, button, or tracking pixel that can be triggered by something other than a genuine customer action, a bot, a CAPTCHA failure, a page reload, a double submission, will happily report a win that never happened.
• Watch for a number that improved for no explained reason. A genuine improvement usually traces back to something specific that changed. An unexplained jump is worth investigating before it's worth celebrating.
• Be willing to correct the record even when the correction is unwelcome. Telling a client, or your own leadership, that a result they were pleased about wasn't real is a harder conversation than reporting a new problem. It's also the only version of the conversation that leads to a decision actually grounded in what's true.

A bad number that's wrong tells you to look for a problem. A good number that's wrong tells you to do more of something that was never working.

Full story, including the demand-ceiling diagnosis alongside this one, on We Told the Client the Market Was Too Small. This is the same underlying discipline covered in Marketing Problem vs. Commercial Problem and Fixing the First Constraint Isn't the Finish Line, and the broader thinking behind it is set out on How Halo Thinks.

What people ask about trusting a good number.

If a number in your own reporting has been trusted without ever being checked, in either direction, a Commercial Diagnostic is a 90-minute session built to find out whether it holds up.