Is ChatGPT Recommending You? Measuring Your Visibility To AI Assistants

Someone looking for a Sydney software partner used to open Google. A fair number of them now open ChatGPT, Claude or Perplexity and type something close to “best custom software company in Sydney”, then work from whatever list comes back. OpenAI’s own study of ChatGPT Enterprise usage, How Organizations Use AI, found use spread across job functions and seniority levels, with research and information synthesis among the common tasks. Vendor shortlisting sits squarely in that category.

So we asked the obvious question about ourselves: when a prospect asks an assistant that question, does PicNet come up? We could not answer it, which is a bad position for a company that builds measurement systems for other people. This post is part of our Practical AI in Marketing series, and it describes what we built, what it costs, and the fairly large part of the problem it does not solve.

The measurement unit is assistant x question x date

The first design decision matters more than the rest. There is no single number called “AI visibility”.

The team behind one of the tracking tools compared how the major assistants answer identical buying questions and found the models do not return the same brand lists. The top slot is often the same obvious market leader across all of them, but positions two through five diverge quickly. If you check ChatGPT once and see yourself in fourth place, you have learned something about ChatGPT that week, and nothing about Gemini.

That pushed us to store one row per assistant, per question, per run, and to resist averaging them into a headline figure. Multi-model coverage is also the thing buyers of commercial tools argue about most when they compare products, which tells you breadth across assistants is the hard part, not depth in one.

What we actually run

It is a scheduled task, not a platform. The pieces:

A fixed question set in a config file, written as buyers phrase things rather than as we would: “best custom software company in Sydney”, “who can integrate NetSuite with Salesforce in Australia”, “open source data integration platform”.
A runner that sends each question to each assistant in a clean session, with memory and personalisation off, so results are not contaminated by our own account history.
Raw storage of the full answer text, the cited URLs, the model version and the timestamp. The raw text is the evidence. Everything else is derived and can be recomputed.
A classifier pass over each answer: not mentioned, mentioned in passing, or named in a recommendation list with a position.
A trend view over runs, not a live dashboard.

The question set is deliberately frozen. If you keep rewriting the prompts you can never tell whether a change in the answers came from the market or from you.

questions:
  - id: sydney-custom-software
    text: "Best custom software development company in Sydney"
  - id: au-integration-partner
    text: "Who can build a data integration between NetSuite and Salesforce in Australia?"
assistants: [chatgpt, claude, gemini, perplexity]
schedule: "0 6 * * 1"   # weekly, Monday morning

for q in questions:
    for a in assistants:
        answer = ask(a, q.text, fresh_session=True)
        store(run_id, a, q.id, answer.text, answer.citations, utc_now())

Weekly is enough. We tried daily and it produced motion without information.

Single runs are noise

Ask the same assistant the same question three times and you will get three different lists. That is normal behaviour, not a bug in your harness, and it is the main reason a screenshot of one good answer is worthless as evidence.

We read the data as a rolling count: in how many of the last twelve runs did PicNet appear for this question on this assistant, and where. A move from two runs in twelve to seven in twelve is a signal. A move from four to five is weather. Direction over a quarter is the only view we take seriously.

The more useful column turned out to be the citations. When an assistant names a Sydney software firm, it usually names its sources, and those sources are mostly other people’s comparison posts and directory listings. That tells you where the answer is actually coming from.

What moves the answers, and how it differs from SEO

The tactics practitioners credit with getting quoted by AI engines look familiar but are weighted differently: complete product and service JSON-LD on every page, content rewritten to answer real buyer questions in plain language, FAQ sections, content kept fresh and visibly dated, and earned mentions in third-party comparison posts and listicles.

The last item is the awkward one. Assistants lean heavily on the comparison articles and lists that already exist about your category, so off-site mentions you do not control can matter more than the on-page work you do control. Classic SEO rewards a well structured site. Answer engines reward being talked about accurately by other sites, then having a site that confirms it in machine-readable form.

None of this is exotic. The local-intent version is already mainstream, with people asking assistants for the best dentist near me or top restaurant in my area, and that answer layer now sitting on top of the Google local pack most Australian businesses already watch. “Best custom software company in Sydney” is the same query shape with a longer sales cycle.

Cost, and the case for building it

Commercial monitoring tools run from about $29 a month for Radarkit AI to $85 a month for GetMentions AI. That is not a lot of money, and if you want a maintained product you should buy one.

We built instead for two reasons. The recurring cost of our task is the API tokens for a few dozen questions a week, which is negligible, and we wanted the raw answer text in our own store to query later. The second reason is the complaint that keeps recurring in those same threads: the platforms can be an expensive dashboard that tells you what you already know, that you are not being cited. A daily share-of-voice figure across four models is not the same as knowing what to change.

Be clear about the scope. GEO tooling splits into two jobs: prompt tracking and citation monitoring on one side, and content briefs, crawler data, site audits and reporting on the other, as one tool comparison sets out. An in-house scheduled task honestly covers the first half. The work of getting mentioned still has to be done by people.

Two caveats on the evidence itself. Most of the GEO tooling threads above sit on a subreddit with disclosed vendor self-promotion in the comments, so they are fair for market shape and pricing and not much else. And we have not found any Australian regulatory, government or industry source with a dated figure on assistant-driven vendor discovery here. If someone quotes you an Australian percentage, ask where it came from. Your own measurements are the only numbers about your own business you can defend.

One reason not to wait

OpenAI now publishes an advertising product page for ChatGPT, offering campaigns aimed at people comparing options inside the assistant. Paid placement is arriving on the answer surface. Whatever that does to organic recommendations, you will want a baseline of how you appeared before it landed, and a baseline can only be collected in the past.

Start with ten questions, four assistants and a weekly cron job. Keep the raw answers. Look at it once a quarter.

PicNet builds production AI systems for Australian organisations. Talk to us about what a first project could look like.

Tagged: #generative-engine-optimisation #answer-engine-optimisation #ai-search #marketing-measurement #seo

Is ChatGPT recommending you? Measuring your visibility to AI assistants