← Writing

Can I trust the Meta algorithm? Perhaps.

I audited $383K of Meta tROAS spend country by country. 7% had been failing on the exact metric the algorithm optimizes and it was hard to catch it.

Somewhere inside a campaign that was hitting its tROAS target, one country had been running at 28% All ROAS for three months. So finding out that the algorithm is clearly slacking off was a shock for me. The target was 70%, no dashboard flagged it, but we found it because we went looking.

This is the story of that audit, and the scoring framework that came out of it.

The setup

A subscription app on Meta. Five tROAS campaigns split by platform and geo tier. March 1 to May 29. $383K in spend across 200+ countries with plenty of ad sets.

Quick vocabulary so we read the same numbers the same way. tROAS means Meta bids toward a total ROAS target. That target is the algorithm’s job description. D0 ROAS is our early signal: in this app, most initial revenue lands on install day, so D0 tells you fast whether a cohort makes sense.

The campaigns were judged the way most campaigns are judged: blended ROAS against target. That number looked on point all quarter.

One blended number hides everything

Here is the Tier1 iOS campaign from that period. Blended D0: 26%. Inside it:

CountryD0 ROASAll ROASWhat was happening
France49%66%Strong, and sliding: All ROAS went from 57% to 40% over the quarter
US23%62%Below target, and the biggest spender in the campaign at $49K
Germany16%31%Below target for three straight months. Going nowhere

Blended D0 says 26%. The countries inside say otherwise.

Three completely different situations. One blended number. And the hard part to accept is, the algorithm didn’t seem to care.

Why the algorithm might not have fixed this

tROAS operates at campaign level. It sees one blended ROAS and adjusts bids across the whole campaign. Per-country ROAS sits outside its optimization loop.

So a bad country hides inside a campaign that looks fine blended. That country running at 28% All ROAS while its campaign printed 70% was failing on the exact metric the algorithm was asked to optimize. Not a proxy. The actual target. D0 flagged it early, All ROAS confirmed it, and the campaign-level number never blinked.

The mechanism is simple: good countries subsidize bad ones. The blend stays mediocre, and mediocre looks like business as usual.

This is not about second-guessing the algorithm. The algorithm optimizes what it can see, which is one campaign-level number. Country performance was a blind spot by design, so someone has to cover it manually. That someone is you. For the record, other possible breakdowns like age or gender looked very normal and expected, so this got us by surprise.

The framework: two metrics, three colors, three buckets

Here is the scoring system we used to separate keep from cut. It runs on a country breakdown export and about thirty minutes a month.

Step 1: score each metric independently

Each country gets two grades, one for D0 ROAS and one for All ROAS, each graded green, yellow, or red against bands built off the campaign’s target. This app’s bands came from its 70% tROAS target: green All ROAS starts at the target, and the D0 bands come from what historically maps to hitting it. Steal the structure. The numbers you derive yourself, from your own target and your own D0-to-All curve.

D0 gives you the initial signal that we look at. All ROAS gives you truth, which is what the campaign is optimizing for. A country at 25% D0 and 80% All is a different animal from 25% D0 and 30% All, and a single metric would blur them together.

Step 2: combine into three buckets

Green scores 2, yellow 1, red 0. Add both metrics.

Sum of both scores Bucket What to do
3 or 4 Good Keep, don’t touch
2 Watch Check monthly
1 or 0 Exclude The algorithm won’t fix it, it doesn’t know it’s broken

Step 3: add the time dimension

Averages hide direction. Two countries can carry the same quarterly verdict and be moving in opposite directions.

PH in the worldwide campaign declined from 33% to 24% to 17% D0, with All ROAS at 28% in a month where the campaign hit 70%. GB dipped to 17% in April and came back to 29% in May, with All ROAS hovering near target the whole time. Same aggregate verdict. Opposite trajectories. The monthly view tells you which one deserves patience.

Step 4: act on it

Exclude the red-reds that stayed red for three months with meaningful spend behind them. Set a spend floor before judging anything: a country with $80 behind it has a noisy ROAS, not a verdict. Watch-list the yellows and re-check monthly. Leave Good alone.

What it surfaced

None of this is visible at campaign level.

46% of spend graded Good. 47% graded Watch. And 7%, or $27.6K across 17 country-campaign combos, had both metrics red for three straight months. The biggest three: PH in the worldwide campaign ($7.1K at 21% D0), MX on Android ($6.2K at 23%), DE in Tier1 ($2.1K at 16%).

That $27.6K stays parked where it is until someone manually looks. No alert fires. Nothing turns red in Ads Manager. The campaign view said everything was fine while it happened.

The takeaway

The algorithm is good at finding buyers. It is not good at abandoning geos that stopped converting.

The whole system fits in four lines:

  • Two metrics: D0 for speed, All ROAS for truth.
  • Three colors per metric, scored against your own target.
  • Three buckets: keep, check monthly, exclude.
  • One calendar reminder. Monthly.

If you’d rather have a second pair of eyes run this audit on your account, my inbox is open.