In January 1995, McArthur Wheeler robbed two Pittsburgh banks in broad daylight. Instead of hiding his face, he had covered it in lemon juice. Lemon juice works as invisible ink, so he reasoned it would hide him from the cameras too. He had even tested the idea: he took a Polaroid of himself, and he wasn’t anywhere in the picture.

In April, his picture ran on the Pittsburgh Crime Stoppers segment of the 11 o’clock news. He was arrested less than an hour after the broadcast. “But I wore the juice,” he said.

The Polaroid is the part worth keeping. He didn’t skip the test. He ran one, and the test agreed with him. The police officer on the case later offered three explanations for the blank photo: bad film, a camera he hadn’t adjusted, or a camera pointed somewhere other than his face. None of them were about lemon juice.

Tracking is full of Polaroids like that.

What the study found, and what it didn’t

The psychologist David Dunning came across the story in the 1996 World Almanac and traced it back to the Pittsburgh Post-Gazette. In 1999 he and Justin Kruger published “Unskilled and Unaware of It”. Across four studies on humor, grammar and logic, the participants who scored in the bottom quartile grossly overestimated how well they had done. On average their scores put them in the 12th percentile, and they estimated themselves to be in the 62nd.

Two details get lost in the retellings. The low scorers didn’t rate themselves higher than the top scorers did; they overestimated the most. And later researchers argued that part of the gap comes from statistics rather than psychology, through the general better-than-average effect and regression to the mean. Kruger and Dunning replied, and the debate continues. The safe reading is a tendency: the skill you need to do something well is often the skill you need to notice you did it badly.

In tracking, that skill is knowing what a check can and can’t prove.

Tracking’s lemon juice

Each of these feels like proof. Each one is a test that can’t fail on the thing you care about, which is whether every paid order reached Meta exactly once.

A green check in Meta Pixel Helper means the browser on that page fired a Pixel event. It says nothing about whether the payment cleared, whether the server sent a second copy of the same order, or whether the next buyer, on another browser with an ad blocker, sent anything at all.

“Active” in Events Manager means events are arriving. A dataset that receives two Purchases for every order is active. So is one that receives half of them.

An AI assistant that reads your GTM export or your webhook code and says the setup looks correct has read your configuration. It hasn’t read your orders, and it can’t see the second Purchase tag someone added last year in a different container, or the checkout platform’s own Meta integration that nobody switched off.

ROAS in Ads Manager counts the purchases Meta attributes to your ads under your attribution setting. It isn’t a count of your orders, and duplicates make it look better. A dashboard that looks great is the one nobody questions.

None of the four counted your orders. That’s the gap.

A table of four common checks (Pixel Helper green check, Events Manager Active status, an AI review, Ads Manager ROAS), what each can prove, and that none counted orders, above one check that compares paid orders with Purchases received.
Only the last row compares Meta's side with a record Meta doesn't control.

The checks that can prove you wrong

A useful check is one with a result you’d be unhappy to see. These are cheap, and each one compares Meta’s side against something Meta doesn’t control.

1. Count paid orders against Purchases received

Take yesterday’s paid orders from your payment processor or store. Exclude test orders and refunds issued the same day, so you’re counting what was really paid. Then take the Purchase count Events Manager received for the same day, in the same timezone. Use Events Manager, not Ads Manager: you want every Purchase Meta received, not the ones it attributed to an ad.

More Purchases than orders usually means duplicates. Fewer means sales Meta never saw. Some gap can be legitimate, for example buyers who declined consent in a region where your setup respects it, so learn your normal gap first. A sudden change in that gap is the real alarm.

If the count is too high, the deduplication guide walks through the usual causes and how to count every sender that reports a Purchase.

2. One test order in Test Events

Place one real test order with the Test Events tab open and the test code on your server events. You should see one browser Purchase and one server Purchase with the same event ID, and one of them marked deduplicated. Two rows with different IDs are two Purchases. Three rows mean a third sender you didn’t know about.

3. A negative control

This is the step most setups skip, and it’s the one that makes the other results trustworthy. Send a second test event with a deliberately changed event ID and confirm it stays a separate event. If it also gets merged, or if nothing changes, your test isn’t measuring what you think it is. A test that can’t fail is lemon juice.

4. Repeat it on a schedule

A setup that passed once can break the day someone edits a tag, installs an app or changes the checkout. Run the order count weekly, and alert on a change in the gap rather than on a single bad day. Silent tracking failures covers what to monitor and where.

Run it on yourself first

The point isn’t that other people’s setups are broken. Wheeler’s problem wasn’t a lack of testing. He trusted a test that couldn’t tell him what he needed to know. Everyone who builds tracking has a version of that, us included.

“I know that I know nothing”, the line usually attributed to Socrates, is a paraphrase of Plato’s Apology rather than a quote. It still works as a working habit: before you trust a setup, name the check that would prove it wrong, then run it.

Sources

  • Errol Morris, “The Anosognosic’s Dilemma: Something’s Wrong but You’ll Never Know What It Is (Part 1)”, The New York Times, June 20, 2010. It quotes the Pittsburgh Post-Gazette’s 1995 coverage.
  • Justin Kruger and David Dunning, “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments”, Journal of Personality and Social Psychology 77(6), 1121-1134, 1999. doi:10.1037/0022-3514.77.6.1121
  • Joachim Krueger and Ross Mueller, “Unskilled, Unaware, or Both? The Better-Than-Average Heuristic and Statistical Regression Predict Errors in Estimates of Own Performance”, Journal of Personality and Social Psychology 82(2), 180-188, 2002.