The One-Week Ad Holdout Test: How to Tell If Your Ads Are Actually Working

05 October 2026·5 min read

Quick answer: A holdout test means deliberately switching your ads off for one full week while you log enquiries day by day, so you can see what volume disappears versus what was always going to turn up anyway. One week won't give you lab-grade data, but it's the fastest, cheapest way to sanity-check whether your ad spend is doing real work or just taking credit for demand you already had. Most businesses never run it because they're genuinely scared of the answer. 📈

Here's a question we ask every new client before we touch their ad account: what happens if we turn it off for a week? The silence that follows is usually the most honest answer we get all onboarding. Nobody wants to find out their "high-performing" campaign is riding on word-of-mouth and people who'd have Googled the business name anyway — but a holdout test uncovers exactly that, and it's a far cheaper lesson to learn on purpose than to stumble into during a budget review 💖. The businesses brave enough to run this test aren't hiding something. They're just confident enough to check.

What most businesses get wrong

They judge ad performance by the platform's own dashboard (impressions, clicks, "conversions") instead of what actually lands in enquiries or bookings.
They never have a genuine "off" period to compare against — every number is only ever relative to another number from the same always-on system.
They pause a campaign for a day or two, call that a test, and draw conclusions from what's really just normal day-to-day noise.
They test during an unusual week — a public holiday, a big sale, a staff shortage — and treat the result as gospel.

The One-Week Holdout Protocol

Step 1 — Pick a genuinely ordinary week. No public holidays, no major promotion, no unusual staffing. The more "normal" the week, the more trustworthy the result.

Step 2 — Log a two-week baseline first. Before you touch anything, record daily enquiries for two weeks so you know what "normal, ads-on" actually looks like, not just a gut feel.

Step 3 — Pause properly, not partially. On day one of test week, fully pause every paid channel being tested — including retargeting and remarketing lists, not just the main campaign. Leave organic, SEO, email and referral channels untouched.

Step 4 — Log every single day. Use this sheet (one row per day):

Date	Phone enquiries	Form/DM enquiries	Total	Notes (source if known)

Step 5 — Resume on day eight, keep logging. Turn ads back on and keep recording for another one to two weeks so you can see whether enquiry volume snaps back to the baseline, overshoots it (a lag effect catching up) or stays flat.

Step 6 — Compare averages, not single days. Average daily enquiries: before vs during vs after. That gap — not any one dramatic day — is your real signal.

A business that's always run ads and never turned them off: Three years of continuous Google Ads, never once paused. The owner assumed most enquiries were ad-driven. The holdout week showed volume dropped by roughly a third, not two-thirds — a big chunk of "ad-driven" leads were going to find them anyway, through search and repeat business.

A seasonal business testing during its slow period: Ran the test deliberately during a quiet month. Enquiries barely moved with ads off — the slow season was slow regardless of spend. That was more useful than "ads work": it meant shifting budget toward the ramp-up period instead of spreading it evenly across a year with a genuinely quiet patch in it.

A booking-based business suspicious its bookings would happen anyway: Strong repeat-client base, with new-client bookings assumed to be "mostly ads." The test isolated the two: returning-client bookings stayed flat with ads off, while new-client enquiries dropped sharply — confirming exactly what the spend's real job was.

How it actually works

A single week is directional, not definitive — and the honest reason is lag. Someone who saw your ad days before test week might still enquire during it, flattering the "during" number. A slow decision-maker who saw an ad during the off week might enquire after ads resume, muddying "after." That's why the comparison matters more than any single day, and why a second holdout a month later is worth far more than treating one week as proof.

Worth separating brand search from direct enquiry too, if you can. If people Googling your business name by name also drops during the test, that's a stronger signal the ads were generating real demand — not just catching people already looking for you.

💡 Run it twice before you trust it. One holdout week tells you something moved. A second one, run four to six weeks later under similarly ordinary conditions, tells you whether that movement repeats. A single data point is a story, not a pattern — and decisions about cutting or increasing spend deserve a pattern.

Mistakes to avoid

Testing during a week with a public holiday, major industry event or unusual promotion — the noise will swamp the signal.
Pausing the main campaign but leaving retargeting running, which keeps warm leads flowing and understates the drop.
Switching off every channel at once if you run several — you'll learn nothing about which one actually earned its keep.
Panicking and switching ads back on by day two because the dip feels uncomfortable — that's exactly where you learn the least.

Frequently asked questions

Will one week give me reliable data?

Directionally, yes — conclusively, no. Day-to-day volatility and lag effects mean one week can overstate or understate the true picture. Treat the first holdout as a hypothesis, and the second as your confirmation.

What if I can't afford to lose a week of leads entirely?

Run a geo-split version instead: pause ads in one service area or suburb only, and compare it against an area where ads keep running.

Should I tell my team before running the test?

Yes, always. Intake staff need to log enquiry source accurately during the test week, and shouldn't panic over a dip that's expected and temporary.

How often should I repeat this?

Roughly every quarter, or whenever cost-per-lead creeps up and you want to check the spend is still earning its keep.

Keep reading 🤍

Written by

Kate, founder of Chronically Online

I help Gold Coast and Brisbane businesses grow with branding, websites and marketing that actually works.

Work with me ✦