AI visibility tools (prompt trackers) promise to measure something that's impossible to measure accurately. To me, they're the business of manufactured certainty.
This technical report looks at why measuring AI visibility is technically problematic, what the available evidence says, and how to evaluate the tools on the market.
My goal is to try to reduce information asymmetry. In other words, to level the playing field so you know the same things that the people selling you these tools, and the agencies offering "GEO/AEO", know (or should know).
Table of contents
What AI visibility is and what prompt trackers measure
AI visibility is how often and how prominently a brand shows up in the answers of language models (ChatGPT, Perplexity, DeepSeek, Google's AI Overviews, Grok, Copilot...) when users ask questions related to its industry.
Prompt trackers are tools that run predefined prompts against these models and measure whether your brand appears, in what position, and how often compared to competitors. They promise to answer questions like: Does ChatGPT recommend me when someone asks for "best email marketing tools"? How often? Above or below my competitors?
The problem is that these questions assume a level of consistency and replicability that LLMs, by design, don't have.
This is something Perplexity's own CCO, Jesse Dwyer, confirms in the WSJ: AI search personalizes results based on the user asking, which makes them hard to predict and measure.
The scientific evidence
LLMs are inconsistent by design
Academic research has shown that language models are inherently non-deterministic, even with supposedly "deterministic" settings:
- A January 2025 paper in ACM Transactions found that in 75.76% of CodeContests tasks, ChatGPT didn't produce two identical outputs with temperature=0, a setting many people wrongly believe guarantees determinism. I showed this in a talk last year.
- Wang and Wang's study (September 2025) analyzed 3.4 million outputs across 50 independent runs per task. The most advanced models aren't more consistent: GPT-4o produced exactly the same answer only 3% of the time, while GPT-3.5 Turbo reached 97%. The authors recommend doing 3-5 runs to improve statistical consistency. Not a single tool on the market talks about this.
- Anthropic's paper (November 2024) revealed a statistical problem that directly affects AI visibility tools: when you run many prompts on related topics, the errors aren't independent. If the model is biased for or against your brand on the topic "CRM software", it will fail systematically, not randomly, across all the CRM prompts. Anthropic found that when questions come in related groups, the real standard error can be more than three times larger, so 30 prompts on similar topics aren't 30 independent data points, but far fewer. There's very little transparency about the exact method/process LLM trackers use to get the answers, so we don't know whether this could be a problem or not.
- Variability doesn't only come from the model's inherent randomness, but also from the prompt format. In my recent review of the scientific literature I documented differences of up to 76 accuracy points from minor changes like spaces, punctuation or capitalization (FormatSpread, Sclar et al.).
Measured volatility: 40-60% monthly change
Profound's study (July 2025) analyzed ~80,000 prompts per platform, comparing June with July:
- Google AI Overviews: 59.3% of the cited domains changed
- ChatGPT: 54.1%
- Microsoft Copilot: 53.4%
- Perplexity: 40.5%
Over 6-month periods, volatility shoots up to 70-90%.

Studies on brand consistency
SparkToro's study (January 2026) ran 2,961 prompts through 600 volunteers using ChatGPT, Claude and Google AI across 12 categories. Results:
- Less than a 1 in 100 chance of getting the same list of brands twice
- Less than a 1 in 1,000 chance of getting the same list in the same order

Variability depends on the size of the consideration set. For queries with few options (Volvo dealerships in Los Angeles), the correlation is high. For broad consideration sets (science fiction novels), variability is extreme.
Rand Fishkin says that although the lists and rankings are random, visibility percentage (how many times a brand appears across multiple runs) can be a valid metric. "City of Hope hospital" appeared in 97% of the answers about cancer hospitals on the West Coast, even though its position kept changing.
My take is that a metric being "less random" doesn't make it useful. Visibility percentage gives you a blurry picture of something you can't control, because it's based on prompts that aren't the ones your real users write, run in conditions that don't reflect their experience. It's the least bad of the indicators, but it's still an indicator of limited value.
There's another problem: visibility has a structural bias toward established brands. Well-known brands (Booking, Amazon, or MediaMarkt, the big European electronics retailer) show more stable metrics because they're overrepresented in the training data. They appear in thousands of different contexts, which creates robust associations in the model's weights. For emerging or niche brands, the same visibility % fluctuates so much between runs that it's practically statistical noise. This is a paradox, because the tools are more "reliable" precisely for those who need them least (brands that already know they're visible), and less reliable for those who'd need them most (brands trying to gain visibility).
To get a "reliable" visibility figure, you need to run hundreds or thousands of prompts multiple times. The computational (and financial) cost of doing that to get a directional data point that doesn't correlate with traffic or conversions is hard to justify.
Mike Sonders' study (February 2026) analyzed 1,200 ChatGPT interactions (12 prompts × 100 runs each, with a different IP per session) focused on B2B software recommendations, and found the following:
- The more niche a category is, the easier it is to show up consistently in AI answers.
- Only ~11% of the brands an LLM mentions in a category show up consistently. The rest rotate almost randomly between answers.
- The more specific the prompt (with a role, context or use case), the harder it is to get into the group of dominant brands, because the model commits more to fewer options. This creates more noise in the measurements, and it's a problem for trackers, because real users write complex, contextual prompts, not the generic "best X software" questions most tools use. A single run per prompt is almost useless given the variability. Sonders recommends a minimum of ~5 runs per prompt in the session. Tracking tools don't do this because they don't know about it, and even if they did, they wouldn't do it, because it would multiply their costs by 5.

How LLMs decide what information to show
The models' QDF system
The leaked GPT-5 system prompt reveals that ChatGPT uses a QDF (Query Deserves Freshness) system on a scale from 0 to 5:
- 0: Historical, being up to date doesn't matter
- 1: Relevant if it's from the last 18 months
- 2: Relevant if it's from the last 6 months
- 3: Relevant if it's from the last 90 days
- 4: Relevant if it's from the last 60 days
- 5: The latest from this month
Not every query triggers a web search. In ChatGPT, a parameter called use_freshness_scoring_profile evaluates, for each query, whether it needs external information. I also found that ChatGPT runs a probabilistic classifier called Sonic that calculates a search_prob (the probability that the query needs fresh data) and compares it against a threshold (force_search_threshold), currently around 65%.
The Claude Sonnet 4.5 leaks show a similar pattern, where the model sorts queries into "never search" (stable facts, definitions), "single search" (unknown terms), and "research" (2-20 calls, cross-checking multiple sources).
Reranking and context signals
ChatGPT's reranking system (ret-rr-skysight-v3) applies additional signals: intent detection, domain terminology and source-type filtering. If internal tools are available (Google Drive, Slack, CRM), the models prioritize them over web search. All of this depends on the context of your conversation.
Google's patents: why synthetic prompts don't work
These two patents show why external tools can't replicate how Google generates query fan-out and organizes results.
- Query Fan-Out (US20230281193A1): When someone searches for something, Google doesn't stick to that literal query. Instead, it generates variants using models trained on the real behavior of millions of users (what they searched for, what they clicked on, what reformulations they made afterwards). It also takes into account things an external tool simply can't know, like where the user is, what they were doing before, or their session history. All of this is processed in real time, adapting to the context of each individual search.
- Thematic Search (US12158907B1): Google analyzes the documents it considers relevant for a search, generates internal summaries of each one, groups them into thematic clusters and then ranks them by criteria like quality, authority or popularity. The problem is that those ranking signals are completely opaque from the outside. We don't know exactly how it weighs each factor or which documents it considers relevant before it shows us anything.
- Google's SAGE (January 2026): Google's deep search agents are being trained to run between 2 and 10 reasoning hops per query, using strategies like resolving conflicts between sources, generating hypotheses and temporal reasoning. All of this happens internally before you see the answer. The synthetic prompts of external tools don't even come close to this complexity.
The empirical data confirms this. Ahrefs' study (March 2026) analyzed 863,000 SERPs and 4 million URLs cited in AI Overviews and found that only 38% of the pages cited in AI Overviews also appear in the organic top 10 for that same query. The remaining 62% comes from pages that rank lower or don't show up at all in the organic results for the original query. In July 2025, that overlap was ~76%. In other words, with Gemini 3, Google is pulling more and more from its internal fan-out to select sources, rather than from the direct results. If even ranking in the top 10 doesn't guarantee you'll be cited, imagine what a prompt tracker can capture by firing queries from the outside, with no access to the internal sub-queries.
Ahrefs' own article acknowledges that AI Overviews are probabilistic, and that their content, citations, entities and fan-outs change with every query.
Why you can't replicate the user experience
Model memory vs. web search
There's a problem I don't think any prompt tracking tool solves: they don't tell you whether the LLM searched the web or answered from its internal knowledge.
*Update Feb 1, 2026: I've found that Waikay.io does do this.
*Update 02/2026: Bing Webmaster Tools has launched AI Performance, which partially solves this problem by reporting only visible citations in Copilot and Bing AI answers. If your URL shows up there, you know it was retrieval, not parametric knowledge. It's the first piece of first-party data that confirms this distinction.
This decision depends on the model's configuration (version, temperature, system prompts, whether search is enabled), on how volatile the topic is and how the prompt is phrased (for prices or news it always searches; for definitions or stable concepts it answers from memory), on its internal confidence threshold (if the model "thinks" it knows the answer, it doesn't search), and on the access method, because the API and the web interface behave differently when it comes to search.
And when it does search, there's another layer of opacity, because the model doesn't run your literal prompt. As we saw earlier, it generates multiple internal subqueries (fan-out), searches in parallel, and synthesizes the results. Your brand may appear in the final answer, but you don't know which subquery was responsible. You can't optimize for something you don't know about.
This completely changes how actionable the data is:
- If the answer comes from web search: you can create and optimize content, do digital PR... but without knowing which exact subquery you're targeting.
- If the answer comes from parametric knowledge: there's nothing you can do in the short term. You depend on the next training cycle and on changing the narrative about your brand in the general corpus.
It's like knowing your portfolio went up 5% without knowing which asset caused it...
And it's not just the subquery you don't know: you don't even know what language it was run in. A Peec AI study (February 2026) analyzed more than 10 million prompts and 20 million query fan-outs and found that 43% of ChatGPT's internal subqueries for non-English prompts are run in English. In 78% of non-English sessions, ChatGPT also runs searches in English. A Spanish user asking in Spanish from Spain gets results influenced by English-language sources. Prompt trackers that fire prompts in Spanish don't capture this extra layer of opacity.
Personalization by context
ChatGPT implements two memory systems (Saved Memories and Chat History) that influence every answer. Experiments by the AEO Agency Team showed that even under "absolutely identical testing conditions, responses sometimes differed".
For location-dependent queries, ChatGPT rewrites prompts internally. A user in New York who asks for "best electronics stores" gets a query processed as "trusted electronics brands NYC store". On top of that, paid users get "significantly better" answers than free users.
Google now takes AI Overviews follow-up questions straight into AI Mode, keeping the conversation context. Search is no longer an isolated query. It's a conversation where each question influences the next one.
Integration with private data
Robby Stein (VP of Product at Google) announced that they'll use data from Gmail, Drive and Calendar to personalize answers. A search for "best hotels in London" will take into account meetings in Calendar, flights in Gmail, budgets in Sheets.
Google launched Personal Intelligence (January 2026), which connects Gemini to Gmail, Photos, YouTube and Search history. The feature uses "context packing" to analyze user repositories that go beyond a million tokens. One of Google's examples: you ask about tires for your car, and Gemini pulls the license plate from a photo in Google Photos, looks up the specs in your Gmail emails, and suggests options based on photos from previous trips.
MCPs and installed tools
Claude, ChatGPT and other LLMs let you connect external tools (MCPs) that change how the model behaves. A user with their CRM's MCP installed will get completely different answers. Visibility tools have no way of knowing which MCPs or what context each user has.
Add to this AI Mode generating interfaces on the fly, group chats in ChatGPT, and memories in every AI... If 50-prompt LLM trackers didn't make sense before, they make even less sense now.
A small change in the system prompt, memories or tools completely changes the result. There's no reliable way to track the real experience of LLM users.
Prompt Trackers: Tools to measure AI visibility
*Update 10/2026: I've updated prices, database sizes and access methods.
The tools on the market differ along four dimensions: scale (how many prompts they run), prompt source (real demand vs. synthetic), access method (API vs. scraping), and business model.
Data scale
Massive scale (millions of prompts):
- SISTRIX: more than 62 million real questions grouped into more than 1.4 million topics. AI Check covers ChatGPT, AI Overviews, AI Mode, Gemini, Perplexity and Copilot.
- Ahrefs Brand Radar: more than 492 million prompts a month: AI Overviews (276.9M), AI Mode (100.4M), Gemini (29M), Copilot (29M), Perplexity (28.9M) and ChatGPT (28.3M). Keep in mind that the mix changes a lot from one month to the next: AI Mode went from 54.8M in May to 16.4M in July and then to 100.4M in October, and Grok disappeared between July and September.
- Semrush AI Visibility: more than 317 million prompts and responses across ChatGPT, Gemini, AI Overviews and AI Mode, with 117 regional databases.
Small scale (tens to hundreds of prompts):
- Otterly AI: 15-400 prompts depending on the plan
- Peec AI: 50 prompts on the basic plan
- Promptwatch: 50-350 prompts depending on the plan
- Profound: a 50-prompt trial and custom Enterprise plans
- Waikay: 40-360 prompts depending on the plan
Prompt source
Real demand (derived from web searches):
- SISTRIX: ChatGPT prompt suggestions + "People Also Ask"
- Ahrefs: a database of 28.7 billion keywords + "People Also Ask"
- Semrush: buys clickstream data from third-party panels (perhaps including prompt data harvested by Chrome or Edge extensions), groups the prompts into topics, anonymizes and simplifies them. So technically it's synthetic, because they aren't the exact prompts
Synthetic (made up by the client or the tool):
- Conductor, Gumshoe, Otterly, Peec, Promptwatch, Profound, Waikay: the client defines their own prompts (for example, using long-tail queries from Search Console) or the tool generates them algorithmically (synthetic prompts, query fan-out simulations)
The difference is whether those synthetic prompts are derived from real demand signals (PAA, keywords with search volume) or are completely made up.
Session setup: Most tools run prompts in "clean" sessions (no cookies, history or memory). This is good for consistency and replicability, but it doesn't reflect the reality of users with accumulated history, saved memories and personalized context. Gumshoe says it solves this by replicating memory, custom instructions and conversation history through the API, using system messages.
Personas/profiles: Gumshoe uses a "persona-first approach", modeling how different user segments (roles, goals, pain points) would phrase their prompts. The rest of the tools treat all queries the same, ignoring the fact that a CTO asks things differently than a marketing manager. However, adding personas introduces another variable: either they don't significantly change the results (according to the Surfer study that tested this with leaked system prompts), or they add another layer of variability that makes the data less comparable across tools.
Access method: API vs. scraping
Surfer SEO's study (December 2025) compared both methods with 1,000 prompts:
- ChatGPT via API: an average of 7 cited sources
- ChatGPT via scraping: an average of 16 cited sources
- Perplexity: 8% source overlap between the two methods
- 8% of API calls fail to detect mentions that do appear in the interface
Wojciech Korczyński's conclusion: "Monitoring API responses as a proxy for your AI visibility is completely wrong."

Tools that document using web interfaces (scraping):
- Ahrefs: they document that all requests are run through the free, publicly available web interfaces of ChatGPT, Gemini, Perplexity and Copilot.
- Otterly: uses Firecrawl.dev and SerpApi as scraping subprocessors
- SISTRIX: uses web scraping exclusively, through real browser interfaces, not APIs. I confirmed this with their CEO, Johannes Beus
- Semrush: documents that the responses "are captured from real requests and not via any APIs of LLMs", although it doesn't say which model it analyzes, only "the latest model of ChatGPT in search mode".
Tools that prioritize the API: Conductor, Gumshoe, Waikay.
Methodology not specified publicly (or not that I could find): Peec, Promptwatch, Profound. I can also understand them keeping it quiet because it breaks the platforms' ToS, so they probably use scraping.
The version lag problem
Tools test model versions that differ from the ones users actually use (the latest available). Many tools are still using GPT-4o, when the reality is that only 0.1% of users currently choose it each day. And Wang and Wang's research also showed that newer models are more variable.
SISTRIX documents exactly which models it tests: gpt-4o-mini and gpt-4.1-mini for ChatGPT; gemini-2.0-flash-lite, gemini-2.0-flash and gemini-2.5-flash for Gemini; v3 for DeepSeek. They update their data when a new foundation model is released, acknowledging that "querying the models more frequently does not add any value, as the information in these foundation models is static."
Most tools don't document which model version they test or how often they update.
Significance thresholds: how many prompts do you need?
The Wang and Wang study I mentioned earlier recommends 3-5 runs per prompt to get statistical consistency, but those runs need to be simultaneous or in the same session. Running a prompt once a day for a month doesn't meet this requirement: that's 30 measurements of 30 different system states (an updated model or backend state, different content retrieved at inference time, a different temporal context), not 30 measurements of the same state.
If there's less than a 1 in 100 chance of getting the same list twice, a tool with 25 prompts can't give you a representative picture of your real visibility, no matter how many times it runs those 25 prompts.
Even so, the real problem isn't so much how often the prompts are run, but scale and representativeness:
- 25 prompts run 30 times each are still 25 prompts. They don't represent the universe of queries real users make about your industry.
- Synthetic prompts made up by the client reflect what the client thinks users ask, not what they actually ask.
SparkToro used 2,961 prompts with 600 volunteers for its study. Profound used ~80,000 prompts per platform. SISTRIX has 62 million questions. The difference in scale determines whether you can detect real patterns or just statistical noise.
Business model and pricing structure
The most relevant difference is whether AI tracking is a standalone product or a feature inside a broader suite.
Specialized tools (tracking as the main product):
- Otterly: from $29/month (15 prompts) to $489/month (400 prompts)
- Peec AI: from €85/month (50 prompts and 3 models of your choice) to €425/month (350 prompts). Each extra model costs €25 to €115 a month
- Promptwatch: a free plan to get started, and paid plans from $95 to $579/month
- Waikay: from $24.95/month (40 prompts a week) to $199.95/month (360 prompts)
- Profound: no longer sells plans to small businesses. In September 2026 it dropped its $99 and $399/month plans, and today it only has a free trial and custom Enterprise plans
- SE Visible: from $99/month (200 prompts) to $355/month (1,000 prompts)
- Gumshoe: $99/month (Starter) or $299/month (Pro) per project
Suites with tracking included or as an add-on:
- SISTRIX: from €119/month excluding VAT, with the AI part "currently free" on all plans (they reserve the right to charge for it later). A database of 62M questions and their own prompt tracking for ChatGPT, Perplexity, AI Overviews and AI Mode.
- Ahrefs Brand Radar: the AI Visibility Index starts at $199/month, with the index of more than 492M prompts and 2,500 custom prompt checks a month, and goes up to $699/month with all models. Custom prompts are also sold separately from $50/month (about 80 a day on one platform), and all paid Ahrefs plans come with a free daily quota: 5 prompts on Lite, 10 on Standard and 20 on Advanced.
- Semrush: the standalone AI Visibility Toolkit costs €94.94/month per domain (25 prompts). Since July 2026 it's also included in the Semrush One plans (SEO + AI): Starter at €190/month (€158.40 with annual billing) with 50 prompts a day, Pro+ with 100, and a custom Advanced plan with unlimited prompts. The SEO-only plans don't include it. A database of more than 317M prompts.
- Conductor: an estimated $10,000-50,000 a year (aimed at Fortune 500 companies)
Running 100 prompts a day with GPT-4o-mini costs ~$2.04/month. Margins at the specialized tools range from 85% to 95%.
As Kevin Indig puts it: "Tracking is a feature, not a company." And his argument isn't that measuring is useless. It's that tracking isn't defensible as a business because it gets commoditized. Amplitude offers LLM monitoring for free, Semrush added it as a checkbox, Sistrix includes it by default... Established platforms absorb tracking as a feature and kill off the standalone tools. That's exactly what's happening.
You might say: "but Ahrefs also started out as a rank tracker." Yes, with two differences. First: traditional rank tracking had a real moat and a proven correlation with organic traffic and conversions. Second: precisely because rank tracking got commoditized, those companies survived by turning into full suites.
Tools that rely solely on AI tracking as their product have incentives to exaggerate its value and to take advantage of the client's lack of knowledge. As I've written before, firing off 50 prompts to "measure your AI visibility" is like running an election poll by asking your friends. Even if they're well chosen and they're 50 prompts specific to your industry, according to Anthropic they'd be the equivalent of far fewer independent data points. You need millions of prompts to extract meaningful patterns. Only tools like SISTRIX, Ahrefs or Semrush, which operate at massive scale, can do that.
What's more, Rand Fishkin called AI answers "statistical lotteries", and this market will reach $200 million in 2026 with "almost zero ROI".
In private conversations with several prompt tracker founders who've asked me for help developing their tools, and whom I haven't named here (some of them already with millions in funding), I've found a worrying pattern: many of them didn't know about basic methodological issues like the difference between synthetic prompts and prompts derived from real demand, API vs. interface, the possibility of buying prompt data, or the statistical limitations I've documented here, among other things. Some admitted to me that "you've got to start somewhere". Others have already pivoted their tool after realizing how big the problem is. In other words, the market is being built faster than it's being understood.
And these days, anyone can build a tracking tool over a weekend. You can even put together your own in no time with open-source FireGEO. I think many AI tracking tools are looking to exploit technical ignorance, FOMO and hype to make money in this moment of uncertainty.
All the tools above are third-party, meaning they run prompts from the outside and try to infer what's happening inside. Bing Webmaster Tools has launched AI Performance, the Search Console of AI, where Microsoft itself tells you which of your content Copilot and Bing AI cited in answers generated for real users. It offers:
- Pages Cited: which URLs on your site were cited in AI answers, with the frequency for each page.
- Grounding Queries: the phrases the system used when retrieving your content. They're not the user's exact prompts, just thematic groupings, but they're the first real way to know which queries you're being cited for.
This wipes out, in one go, several problems that affect external tools: there's no version lag (it measures the model real users use), no API vs. scraping problem (it's the system's internal data), no synthetic prompts (it reflects real activity), and it separates visible citations from internal grounding (it only reports what the user sees). Real data, from the real system, about real users.
Important limitations:
- It only covers Microsoft Copilot and Bing AI.
- The data is aggregated and sampled, not a full log of every citation, so very low activity may not show up.
- Grounding queries are generalized phrases, not the exact prompts users typed.
- It doesn't measure traffic, clicks or conversions. A citation doesn't mean the user did anything with your link.
- It doesn't calculate rankings, authority or visibility scores. It just counts citations, and that's not bad at all.
*Update 10/2026: I've analyzed the grounding queries of 6 sites with very different profiles, from a professional's website with 13 cited pages to online stores and transactional platforms with almost 5,000. I exported them with this bookmarklet, because the native export doesn't include the relationship between query and page, and I cross-referenced them with the Google Search Console and Bing queries for the same period. I count a grounding query as "already showing up" when that same query has impressions in either of the two. Between 67% and 93% of the grounding queries were already showing up in one of the two search engines, with a median of around 76%. The ones that only exist in grounding make up between 7% and 33%, and if you weight them by citations, they drop to between 0.4% and 20%. Here are the tables for each site.
This has an important implication: you don't need synthetic prompts or to guess how each user writes, which is impossible anyway. Whatever question the user asks, the system turns it into shorter subqueries (the fan-out), and those subqueries look a lot like the searches you already see in Search Console. With your first-party data, you already cover about three out of four queries, and that's enough information to make informed decisions.
Google has taken the first step in the same direction. In June 2026 it announced a generative AI performance report in Search Console, which has been available to sites worldwide since August 31 (although not every property has it). It lumps AI Overviews and AI Mode together into a single figure and only shows impressions by page, country, device and date. No clicks, no queries. It's less than what Bing gives you, but it confirms the direction things are going. OpenAI could do the same for ChatGPT with search ads. Until that happens, Microsoft is the only one providing first-party data with queries: the grounding queries in Bing AI Performance and the ones in Clarity, which has included them in its citations report since May 2026.
Why current AI visibility metrics don't work
Visibility score
Franco warns that "there is no tool on the market with 100% accurate insight into what users are typing into AI tools. That means any visibility score reported by third-party tools is modeled, not measured."
The fundamental problem is that each tool uses a different formula.
- Geneo calculates: presence × 0.5 + prominence × 0.3 + expected clicks × 0.2
- Advanced Web Ranking assigns: position 1 = 100%, position 2 = 90%... position 10 = 10%.
- Surfer uses brand mentions ÷ AI answers analyzed.
- RankForLLM scores 7 categories from 0 to 5, for a total of 35 points.
Four completely different formulas. A visibility score of 45 in one tool means nothing compared to a 45 in another. And none of them tells you whether that number is "good" or "bad", because there's no universal benchmark to compare it against. You might say: "well, what matters is the trend, so just stick with one tool". I've worked with several tools and gotten different trends, both for my client and for its competitors. There's no correlation between them.
Share of Voice
Cassie Clark documents that this metric doesn't assign weight by position or prominence, doesn't correlate with clicks or conversions (AI answers satisfy the intent without requiring a click), and doesn't combine multiple prompts into a composite score. In other words, it measures presence, not impact.
Even LinkedIn, with access to proprietary data on LLM traffic to its own websites, acknowledges that the challenge lies in the dark funnel and that they can't quantify how visibility within LLM answers impacts the bottom line.
But there's a more basic problem: the calculation depends on who you define as a competitor. The typical formula is: your brand's mentions ÷ total mentions (your brand + competitors). If you add a new competitor to your list, your Share of Voice automatically goes down without anything real having changed in the market. If you remove one, it goes up. The denominator is completely arbitrary.
On top of that, SoV varies significantly between platforms, since a brand can have 20% on ChatGPT but only 10% on Copilot. What's your "true" Share of Voice? There's no answer.
In traditional advertising, Share of Voice has a proven relationship with market share. In LLMs, that correlation hasn't been proven, let alone the connection to business outcomes.
Ranking position
The concept of "position" in an AI answer is meaningless. In a 500-word answer that mentions 5 sources, who's in "position 1"? The first source mentioned? The most cited one? The one that appears in the first paragraph? SEO Testing confirms that "because LLM outputs often vary by session, model, and exact prompt phrasing, there's no universal 'Position 1' like there was with keyword tracking."
As we saw in the SparkToro study, there's less than a 1 in 1,000 chance of getting the same list of brands in the same order. Position is essentially random. However, the appearance rate (how many times a brand shows up across multiple runs) can be more stable and potentially useful as a metric.
Prompt volume
Several tools show "AI Search Volume" or "Prompt Volume" metrics that look like real demand, but they're completely modeled.
Semrush shows "Related Topics AI Volume", which it defines as "estimated monthly AI search activity". In other words, an estimate, not real data. Ahrefs is more honest: it acknowledges that nobody has reliable prompt volume data, because an outside tool can almost never capture users' real prompts.
This search volume problem is worse than in traditional SEO because:
- In Google, search volume is based on real query data (although aggregated)
- In LLMs, nobody outside OpenAI, Google or Anthropic knows what users ask
- Tracked prompts tend to be more specific and long-tail than traditional searches, so even if there were data, the volumes would be lower
- Monitoring itself drastically changes the result: running the same prompt from a tool's account inflates the volume disproportionately.
Citations
Disambiguation
There's a technical problem that affects all the metrics above: brand detection. "Stradivarius" can be the Spanish fashion brand or the violin (or its maker). "Apple" can be the tech company or the fruit. "Corona" can be the beer, the virus, the sun's corona or, in Spanish, a dental crown or a royal one... you get my point?
Most tools don't do entity disambiguation. They do simple string matching, which means they check whether "Stradivarius" appears in the answer without analyzing the context. This creates false positives that artificially inflate visibility metrics. If your brand shares its name with something popular in another field, your visibility score may include mentions that have nothing to do with you.
SISTRIX is actively working on contextual brand detection to solve this problem, and Semrush says it uses its own brand extraction system that takes context into account (it can tell Tesla, Nikola Tesla and Nikola Tesla Airport apart), although neither of them publishes error rates. It's a limitation most tools don't even mention.
Hidden links
There's another technical problem that directly affects the reliability of citation metrics. RESONEO identified three different types of URLs in ChatGPT's code:
- Citations: visible links at the end of sentences + the sources panel. Maximum visibility for the user.
- Other Sources: the "More" section below the citations. Medium visibility.
- Hidden Links: URLs that ChatGPT uses internally for grounding but that are never shown to the user. Zero visibility.
As RESONEO's Olivier puts it: "Studies showing Arxiv or YouTube as 'top cited domains' likely failed to distinguish between real citations and invisible internal grounding URLs."
My bet: Large-scale prompt tracking
First of all, I want to make it clear that even the tools with millions of prompts have a fundamental problem, because each one uses a different corpus. SISTRIX has 62 million questions, Ahrefs more than 492 million prompts and Semrush more than 317 million. But they're not the same prompts.
This means your brand's "visibility" in SISTRIX isn't comparable to its visibility in Ahrefs or Semrush. Each tool will give you a different number because they're measuring different universes. There's no way to know which one is "closer to reality", because none of them has access to users' real prompts.
Another problem with these tools is that for niche clients or little-known brands, you'll most likely find nothing in them.
Topic GAPs
That said, to me these massive-scale tools are useful for one specific thing: spotting possible visibility gaps by topic. If Ahrefs shows you that your competitor appears in a cluster of prompts about "accounting software for the self-employed" and you don't, that's a signal worth looking into, regardless of the exact visibility number. Clustering by topic lets you identify areas where your content isn't being cited, even if the absolute "score" is arbitrary. Sometimes I've found that the cluster really wasn't covered or, if it was, the content had accessibility or structure problems. You can read more about this here:
- Chunking content for AI and SEO
- Web agents and the importance of semantic, accessible HTML
- HTML content extraction in LLMs with a search tool
- Agentic traffic: how AI agents browse and what you need to know
- SEO for AI: A practical guide to getting your small business cited by LLMs
- It's not GEO or AEO, it's just SEO: how to do SEO for AI
But it's also true that if that topic wasn't covered, you could have spotted it with a good old keyword GAP. I've done a lot of prompt gaps with Brand Radar, and the result looks a lot like a classic keyword gap. The recommendations end up being practically the same. That makes sense, because SEO visibility and AI visibility go hand in hand. SISTRIX has measured this in German e-commerce: across about 2,500 online stores, the rank correlation between SEO visibility and mentions in AI answers is 0.66.
However, there's one case where these tools do offer something the traditional keyword gap can't: when the citation doesn't come from your domain, but from a third party that mentions you.
Look at this example: for the query "mba abroad" from India, IESE (the Spanish business school) is mentioned in the AI Overview. But if you check the 9 cited sources, none of them is iese.edu. They're third-party sites that mention IESE in their rankings or comparisons.

This means IESE's visibility in this context depends on its reputation in the ecosystem, not on its own content. A keyword gap comparing iese.edu against esade.edu (another Spanish business school) would never show you this, because you're comparing your own URLs, not external mentions.
In that sense, these tools work more like mention monitoring contextualized by intent than like a classic SEO analysis. They let you see where your brand appears (or doesn't) in the "knowledge graph" the LLM has built from multiple sources, not just from your site.
And for me, this is the real value of a large-scale prompt gap: understanding for which types of content a citation turns into a mention, and for which it doesn't. They're two different decisions. The citation is decided by the retrieval system, which looks for evidence. The mention is decided by the model, based on what it already believes about your brand. If the LLM cites your URL but recommends another brand, your content works and the problem is the brand. If it mentions you but cites third parties, your brand is strong and others are providing the evidence. Knowing why you weren't chosen as a mention when you already had the citation explains a lot, and it's the basis of my (in)visibility framework, which I explain in the presentation at the end of this article (slide 23).
Tools like Sistrix or Ahrefs also let you monitor your own custom prompts. Personally, I don't find that useful for the reasons I've explained, but let's be realistic: there are always clients whose boss's boss asks them for "the visibility report for X prompts". Even if you know the absolute metrics aren't reliable, at least these tools cover that need without you having to build something from scratch.
I also like their honesty:
- SISTRIX acknowledges that "the data situation for LLM/AI analysis is currently even more challenging than for Google or Amazon." They compare the current moment to the "early days of Google analysis". They offer their AI module as a free beta without promising optimization results. In fact, SISTRIX's CEO has confirmed to me that they're moving away from the concept of "single prompt rankings" because they lack statistical validity given the high volatility, and that they're working on Prompt Clustering as a strategic approach
- Ahrefs Brand Radar documents its limitations: nobody has reliable prompt volume data, because an outside tool can almost never capture users' real prompts.
First-party data
Before Google Search Console, the market also relied on external tools that estimated rankings and traffic. The arrival of first-party data didn't eliminate external tools, but it did redefine their role: they went from being the main source to being a complement for competitive analysis.
Google has already started extending Search Console to AI Overviews and AI Mode, although for now only with impressions. When it adds queries, and if OpenAI offers something similar for ChatGPT search, the value of third-party tracking tools will shrink even further, down to what I've already argued for: competitive topic-gap analysis at massive scale, not measuring "your" visibility with synthetic prompts.
Conclusion
The AI visibility tracking tool market is an industry built on questionable foundations. LLM answers are, by design, probabilistic and personalized (the opposite of what you'd need for accurate measurement).
I'm not arguing that we shouldn't measure. I'm arguing that you need to understand what you're measuring, with what limitations, and at what scale it makes sense.
Rankings are essentially random. Aggregate visibility percentage can have some validity if you run enough repetitions, but APIs give different results than the real interfaces, personalization makes it impossible to replicate the user experience, and no tool has access to the real searches users make in LLM chatbots. The internal processes of AI chatbots like Google's (query fan-out, reranking, SAGE) are technically impossible to replicate from the outside, most tools don't distinguish between internal knowledge and retrieval (which determines whether you can act or not), and the visibility % is, paradoxically, more stable for brands that are already well known and need it least.
With gross margins of 85-95%, the value vendors capture far exceeds the value they deliver to clients. This market sells certainty where there's only uncertainty. Anyone buying these tools should know they're getting weak indicators, and by no means absolute truths.