How to Vet an AI SEO Agency: 15 Questions to Ask Before You Sign

Hiring an AI SEO agency is high-stakes precisely because the category is new, unregulated, and crowded with newcomers who repackage old tactics or promise things no one can deliver. The good news is that fifteen sharp questions will separate the agencies that understand AI search from the ones improvising, long before you sign anything.

This guide is for founders and marketing leads about to hire help with answer engines and generative search, and it gives you the exact questions plus what a good answer sounds like. The pressure to act is real: the Gartner search forecast projects traditional search volume falling 25% by 2026 as buyers shift to AI, so getting this hire right matters more every quarter.

If the discipline itself is still fuzzy, our explainer on what AI SEO is is worth reading first, because you vet an agency far better once you understand what you are actually buying. The fifteen questions below are the ones that reliably separate genuine specialists from confident opportunists, and each comes with the answer a strong agency gives and the answer that should make you pause.

Why Does Vetting an AI SEO Agency Matter?

Because the cost of a bad hire is not just the fee, it is the months of lost visibility while a weak agency produces content no engine will ever cite. AI search is new enough that few buyers can tell a genuine specialist from a confident generalist, which is exactly the gap opportunists exploit.

The stakes keep rising. Bain and Company estimates AI is already cutting organic web traffic by 15% to 25%, so the visibility an agency is supposed to protect is genuinely eroding while you deliberate.

That erosion is the same pressure we cover in whether AI is killing website traffic, and it is exactly why a careful hire beats a fast one. Spend an extra week vetting and you risk very little; sign with the wrong agency and you can lose two or three quarters you will not get back, plus the fee.

Quick verdict: Vet on substance, not slides. The agencies worth hiring can explain how AI engines actually pick sources, show you how they measure citations, itemize what you get each month, and tell you honestly when you are a bad fit. The ones to avoid lead with guarantees, hide their metrics, and cannot say who does the work. Ask the fifteen questions below on a call, and the gap becomes obvious fast.

What 15 Questions Should You Ask Before You Sign?

Ask these fifteen, grouped into five themes, and judge the answers against the signals noted. You are testing for genuine understanding, transparency, and honesty, in that order.

Strategy and Understanding

These first three test whether the agency truly grasps AI search or is rebranding old SEO, which is where real answer engine optimization fluency shows itself in minutes.

1. Which AI engines and surfaces do you optimize for, and how does your approach differ for each? A strong answer names ChatGPT, Google AI Overviews, Gemini, and Perplexity and explains that each one retrieves and cites sources differently, so the tactics are not identical across them. The red-flag answer is a confident but generic “we do AI SEO” with no per-engine detail, which usually means a single playbook applied everywhere and adjusted nowhere.
2. How do you define and measure AI visibility, and how is that different from rankings? You want an agency that talks about citations and mentions inside AI answers, tracked across engines, not only keyword positions on the classic results page. If the entire pitch still rests on climbing traditional rankings, you are buying old SEO with a new label on the invoice.
3. How do AI Overviews and answer engines actually decide which sources to cite? A credible answer references being indexed and snippet-eligible, strong E-E-A-T, corroboration across the web, and the plain fact that no special markup forces inclusion. Anyone selling a proprietary trick or secret schema that supposedly guarantees citations does not understand how these systems actually work, and is telling you so.

Method and Deliverables

These expose whether there is a real program behind the pitch, the kind of structured approach our AI SEO framework lays out, or just billable motion dressed up as strategy.

4. What exactly is included every month, in writing? You want named deliverables: a set number of content pieces, entity and schema work, technical fixes, and reporting, with rough proportions you can hold them to. “Ongoing optimization” or “content as needed” with no itemized scope is how retainers quietly bill for activity instead of progress.
5. How do you approach entity SEO and structured data? Look for a clear explanation of making your brand a recognized entity through consistent naming, author credentials, and markup that ties you to your verified real-world profiles. If they cannot describe entities beyond “we add some schema,” their grasp of AI search is thinner than the pitch suggests.
6. Who writes the content, and is it reviewed or signed by a credible expert? The standard to hold them to is Google’s own: the Google helpful content guidance asks for demonstrated expertise and clear authorship, which is exactly what earns trust and citations on serious topics. Anonymous, high-volume content from an undisclosed offshore mill is a red flag, and a serious one in regulated fields.

Measurement and Reporting

Measurement is where weak agencies hide, so press hardest here. A structured AI visibility diagnostic is the kind of baseline a serious agency starts from, and these three questions reveal whether they think that way or just send you screenshots.

7. What will you measure and report, and how often? You want AI citation tracking across engines alongside traffic and conversions, on a fixed cadence you agree upfront. A red-flag answer is a monthly report of rising impressions that carefully avoids whether any AI engine actually names you, because that is the metric that is hard to fake and easy to ignore.
8. What baseline will you record before starting? A serious agency documents where you stand today, which prompts already cite you and which do not, so progress is provable in ninety days. No baseline means no honest way to prove what they changed, which conveniently also means no way for you to hold them accountable later.
9. When should we realistically expect to see movement? A trustworthy answer is months, not days, and it is grounded in how the work compounds: the arXiv GEO research paper by Aggarwal and colleagues shows citable, well-sourced content lifts visibility, but that lift builds over time. Anyone promising fast rankings or instant citations is either inexperienced or being dishonest about how AI search works.

Trust and Track Record

These four test whether you can trust the people, not just the pitch, and they are the questions vague agencies most want to skip past.

10. Can you show results or references in my industry? Ask for case studies or reachable clients in a similar vertical, and pay attention to specifics rather than a wall of logos. Refusal, or only vague and unnamed success stories, should lower your confidence, because real results usually come with details an agency is glad to share.
11. Who actually does the work, in-house or outsourced? You are hiring for expertise, so you deserve to know whether the strategists on the call or an anonymous subcontracted team will run your account day to day. A small, lean team is perfectly fine; an evasive answer about who touches your work is not.
12. Do you understand my industry’s compliance rules? For law, finance, and healthcare especially, the agency must respect bar, SEC, or other advertising rules in everything it publishes for you. An agency that waves compliance away is a liability, because the exposure from a bad or misleading claim lands on you, not on them.

Commercials and Risk

The last three protect your budget and your exit, and the final one is the most important question in this guide.

13. What is the contract length, and what are the exit terms? Favor month-to-month or a short initial term until they prove value, with a clean, written way out that does not require a lawyer. Long lock-ins demanded before any results exist to protect the agency’s revenue, not to serve your outcome.
14. What is your pricing model, and what drives the cost? You want a clear model tied to scope, the kind our guide to how much AI SEO costs breaks down, rather than a single mystery number with nothing behind it. How transparent an agency is about price usually predicts how transparent it will be about results.
15. Do you guarantee specific rankings or AI citations? The only honest answer is no, because no agency controls Google or the AI models, and outcomes depend partly on your own site and market. A confident guarantee of rankings or citations is the single biggest red flag on this entire list, and on its own it is reason enough to walk away.

What Are the Biggest Red Flags in an AI SEO Agency?

The clearest red flag is any guarantee of specific rankings or citations, followed closely by an agency that cannot explain how it measures AI visibility. Both signal either inexperience or a willingness to mislead, and neither is worth your budget.

A few others recur. Watch for a refusal to itemize scope, a pitch built on a secret method rather than fundamentals, no baseline or citation reporting, undisclosed offshore content production, and long contracts demanded before any proof. The category has drawn opportunists precisely because the shift is real, and SparkToro zero-click research found fewer than a third of Google searches now send a click, so demand for help is high and buyer scrutiny is often low. Slow down wherever the answers get vague, and treat confidence without specifics as the warning it is.

One subtler red flag deserves its own mention: an agency that talks only about Google and ignores ChatGPT, Gemini, and Perplexity entirely. AI search is not just AI Overviews, and a genuine specialist will have a clear view on every major engine your buyers actually use. If every example they reach for is a classic blue-link ranking, you are probably hiring a traditional SEO shop that added the words AI search to its homepage but not to its methods.

How Do You Choose After the Calls?

Choose the agency whose answers were most specific, most measurable, and most honest, even where honesty was inconvenient. After the calls, line up how each defined AI visibility, what they itemized, whether they recorded a baseline, and how they handled the guarantee question. The one that told you plainly when you would be a poor fit usually deserves the most trust.

Be honest about fit on your side too. If you have no site foundation, cannot produce or approve content, or need results next week, no agency can help yet, and a good one will tell you so rather than take the retainer. The right hire is the one that scopes to your situation and can show you where you stand before promising anything.

A simple scorecard makes the choice clearer. Rate each agency one to five on the same handful of things: clarity of how they define AI visibility, specificity of the monthly scope, quality of their measurement and baseline, relevance of the track record, and honesty on guarantees and fit.

The agency that scores highest on honesty and measurement, not the one with the slickest deck or the boldest promise, is almost always the right call. If two are close, the tiebreaker is easy: which one told you something you did not want to hear? That candor is the best predictor of how they will treat you once the contract is signed.

Want to see what good answers to these fifteen questions actually sound like? Book a strategy call and put them to us directly, no pressure and no guarantees, just straight answers.

Frequently Asked Questions

What should I ask an AI SEO agency before hiring them?

Ask how they optimize for each AI engine, how they define and measure AI visibility versus rankings, exactly what is included monthly, who does the work, what baseline and reporting they provide, their contract and exit terms, and whether they guarantee results. The last one matters most: any guarantee of rankings or citations is a red flag.

What is a red flag when hiring an AI SEO agency?

The biggest red flag is guaranteeing specific rankings or AI citations, since no one controls Google or the AI models. Others include refusing to itemize scope, hiding measurement behind vanity metrics, selling a “secret” method instead of fundamentals, undisclosed offshore content, and demanding a long contract before showing any proof of value.

How do you measure whether an AI SEO agency is working?

A good agency records a baseline before starting, then tracks whether AI engines like ChatGPT, Gemini, and Google AI Overviews cite your brand, alongside traffic and conversions, reported on a set schedule. If they only show rising impressions or rankings with no citation data, you cannot tell whether the work is actually landing.

How long until an AI SEO agency shows results?

Realistically months, not days, because AI visibility compounds as content, entity signals, and corroboration build. A trustworthy agency sets that expectation upfront and records a baseline so progress is provable. Any agency promising fast rankings or instant citations is either inexperienced or being dishonest about how AI search works.

Should I hire an AI SEO agency or do it in-house?

It depends on capacity and expertise. If someone on your team can run content, entity work, and cross-engine measurement each month, in-house or tools may suffice. If not, a scoped agency delivers more. Either way, vet for understanding and honesty first, and never sign a long contract before an agency proves it can measure and move visibility.

Resources & Further Reading

The following authoritative sources were used to inform and validate this article:

• Gartner Search Forecast – forecast that AI chatbots will cut traditional search volume 25% by 2026
• Bain and Company Research – estimate that AI is reducing organic web traffic by 15% to 25%
• Google Helpful Content Guidance – Google’s standard for expertise, authorship, and people-first content
• arXiv GEO Research Paper – study showing citable, well-sourced content lifts visibility in generative answers
• SparkToro Zero-Click Research – research finding fewer than a third of Google searches send a click