Most owners who ask this already suspect the answer. They've paid an agency or a freelancer for a year or so. The monthly report arrives, it has charts in it, and they still can't tell whether the phone rings more than it used to.
That's a bad spot to be in. It isn't proof the retainer is failing, though. More often it means nobody agreed what "working" meant in the first place.
So before you cancel, or renew, answer these five questions. You'll need your last three reports, your invoices and about an afternoon.
1. Can you say, in one sentence, what you're paying for?
Not the channels. The outcome. "More leads" doesn't count. "Booked estimates from homeowners in Durham Region, tracked by where they came from" does.
If you can't write that sentence, there's a fair chance the agency can't either. So they fill the gap with activity: posts published, keywords tracked, impressions served. None of that is bad. None of it is the thing you're paying for.
Find the original proposal or contract and read the scope. If it lists deliverables ("8 social posts a month, a monthly SEO report") and no business goal, you've found the first problem. It's also the cheapest one to fix. One conversation, one rewritten paragraph.
2. Does the report connect the work to money?
A useful monthly report answers three things, in this order: what happened to the numbers you care about, what was done, and what changes next month. Most reports only do the middle one.
| What reports usually show | What you actually need to see |
|---|---|
| Impressions and reach | Enquiries, broken down by source |
| Clicks and click-through rate | Cost per enquiry, and per booked job or sale |
| "Rankings improved for 14 keywords" | Which pages bring enquiries, and which bring none |
| Posts published, tasks completed | What was tested, and what was learned |
| Next month's content calendar | What changes next month, and why |
If your report lives entirely in the left column, ask for the right one. A competent agency can produce it within a month or two, provided the tracking exists. If they say it can't be done, ask why. Sometimes the honest answer is that call and form tracking were never set up. That's worth knowing too, and it leads straight to the next question.
3. Is the tracking actually there?
This is the unglamorous one, and it's where I look first. Check:
- Do form submissions show up as key events in Google Analytics 4 (or whatever you use)?
- Are phone calls from the website and from ads tracked, even roughly?
- Does your Google Business Profile get its own line in the report? For a lot of local businesses it produces more calls than the website does.
- Do the ad platforms and GA4 roughly agree, or does each one claim credit for everything?
- Does someone on your team record where new customers heard about you, at the moment they book?
That last one is free. A single "How did you hear about us?" question at booking, filled in honestly for 90 days, will tell you more than most dashboards.
4. Who owns the accounts?
Try logging in to your Google Ads account. Then your GA4 property, your Meta Business Manager and your domain registrar.
If you can't, because they sit under the agency's login, fix that first, whatever you decide about the retainer. You paid for those ads and that data. If the relationship ends badly you don't want to rebuild from nothing.
Ask for admin access to each one, in your own name. Good agencies hand it over without fuss. Resistance tells you something.
5. Would you notice if it stopped?
Picture the retainer ending on the last day of this month. What would you lose over the next 90 days?
If the answer is "the Google Ads campaigns would stop and enquiries would drop," the retainer is doing real work, even if the reporting is weak. If your honest answer is "I'm not sure anything would change," take that seriously.
What the answers usually point to
- The work seems fine but the reporting is weak. Keep the agency. Rewrite the scope and the report format together, then give it 90 days under the new reporting before you judge.
- Tracking is missing or broken. Fix measurement before anything else. You can't judge any channel without it, and switching agencies won't fix it for you.
- The accounts aren't in your name. Get admin access now. Decide about the retainer afterwards.
- Lots of activity, no goal, no plan. The problem sits upstream of the agency. They need a plan to execute and someone who holds them to it. That's a common situation, and it often isn't the agency's fault.
Give it a fair timeline, but not forever
Don't judge a new retainer on month one. Month one is setup and diagnosis. Months two and three are when the first changes go live. SEO work often needs a couple of quarters before it shows much, while paid search can show movement within weeks, because you can see what the money bought.
"Give it time" has a limit, though. If you're six months in and still can't answer questions 1 and 2, time isn't what's missing.
What I'd do next
Tell the agency you're reviewing the relationship, and share these five questions with them. Most will take it well. Ask for the report in the new format by next month and admin access by the end of this week. Then decide, with evidence in front of you.
If you'd like an outside read before that conversation, that's what my marketing audit is for. I review the website, tracking, ads, Google Business Profile, email and social, write it up, and give you a prioritized action list and a 90-day plan. I'll also tell you plainly whether the agency is doing good work. Sometimes it is, and the fix is in the brief. You can see the format in an illustrative audit excerpt written for a fictional company.