The short answer
For an AI assistant to cite a UK professional services firm, four things must hold in order: the operator's crawler is allowed to read the site, the page is in its index, the answer is on the page as text, and the page answers the question as a client would ask it. A managing partner can check the first three in ten minutes. No check shows that an assistant will name the firm.
Key takeaways
- Citation is a chain of four links, and a break at any one ends it.
- Each operator runs its own crawlers, so permission has to be checked operator by operator.
- Google says AI features need no extra technical requirements beyond being indexed and eligible for a snippet.
- If a sentence cannot be selected and copied on the page, treat it as missing.
- Nobody outside the operator can promise a citation, so a promise is a warning sign.
Imagine a managing partner at an eighteen-person UK practice, invented for this example. She types her own firm's name and a plain question into an assistant, sees a competitor named and her own practice absent, and concludes that visibility is something to buy. Within a week a supplier agrees with her and quotes. Yet she has not established which of four separate conditions failed, and three of them are visible from the firm's own website. The supplier may be right. The point is that the quote arrives before the diagnosis, and the diagnosis is cheaper.
Four links, and a break in any of them ends the chain
An assistant can cite a page only if the page survives four steps, and the steps run in order. The table lists each one with what a partner opens to check it and who can fix it. The first three are matters of fact on the firm's own site. The fourth is a matter of judgement, which is why it comes last.
| Link | What the partner opens | What a break looks like | Who can fix it |
|---|---|---|---|
| 1. Permission | yourfirm.co.uk/robots.txt, typed into the address bar | A line refusing the operator's crawler, or a refusal applied to every crawler | Whoever holds the site and hosting account |
| 2. Index | The firm's own page in ordinary search results | The page does not appear, or appears without a description | The site owner, often through the agency that built it |
| 3. Text | The page itself, with the key sentence selected and copied | The sentence sits in an image, a scan or a downloadable file only | Whoever edits the page |
| 4. Match | The question a client would type, set beside the page | The page answers a different question, or none | The partner who knows the answer |
Permission is set per operator
Each operator publishes its own crawlers, with different purposes and separate settings. That is why a single yes or no on “do we block AI” has no meaning. A firm can refuse a crawler that gathers training data and still be readable by one that supports search, and the file that records the choice is plain text any partner can open. The matrix sets out what each operator's own documentation says, and nothing beyond it.
| Operator | What it publishes | What the firm checks |
|---|---|---|
| OpenAI | Separate crawlers for training and for search | Whether the search crawler is named in a refusal |
| Anthropic | ClaudeBot for training, Claude-User for user requests, Claude-SearchBot for search quality | Which of the three the file refuses, if any |
| Perplexity | PerplexityBot for its index and a separate agent for user questions | Whether PerplexityBot is refused |
| No separate requirement: the page must be indexed and eligible for a snippet | Whether the page shows in ordinary results with a description |
OpenAI's crawlers are described on its bots page, Anthropic's on its crawler help article and Perplexity's in its crawler documentation. Google's position is in its guidance on AI features, which says a page needs to be indexed and eligible to appear in Search with a snippet, with no additional technical requirements. That makes ordinary search visibility the entry condition for Google's answers, and it is the one a practice most often already meets.
Index and text are the quiet failures
A page can be allowed and still be unreadable, and the commonest cause is that the answer is not text. Many UK firms publish fee guides, service descriptions and team profiles as a downloadable file or as an image of a document. A human reads them without trouble. The test is mechanical: open the page, drag across the sentence that answers the client's question and try to copy it. If it will not select, treat it as absent. This is a proxy, not a claim about what any assistant can or cannot read, but it costs ten seconds and it points at the page that most needs rewriting.
The index check is as simple. Search for the firm's name and the practice area together, and look for the specific page rather than the home page. A service page that never appears has a problem before any assistant is involved, and it is a problem the firm's existing web supplier should be asked about first. When the firm does brief someone, the habit of naming one service and one kind of enquiry, set out in what a tax advisory firm should brief an agency on, keeps the request specific.
Before you buy visibility work
Want a specialist who starts from the broken link?
Free to your firm. The agency pays us, and only if the relationship works, so we have no reason to favour a supplier who sells the same package to every practice.
Get a recommendationThe ten-minute check, worked through
The check is a short sequence, and the order matters more than the speed. Back to the invented practice. She opens the robots file and finds no refusal on a search crawler, so link one holds. She searches the firm name with its main service and finds the home page but not the service page, so link two is the break. Further checking is wasted until that is fixed. Had the page appeared, she would have copied the fee sentence and found it locked in an image, which would have made link three the break. The figures and pages here are invented to show the method and describe no real firm.
Write down the result for each link on one line with the date. Repeat the check after any change to the website, since a redesign or a move of hosting can silently undo a setting. A short written record also gives a supplier a precise starting point, and it settles whether a later improvement came from their work.
Reading a robots file without technical help
The file is a list of names and instructions, and a partner can read it as plain text. Each block starts with a line naming a crawler, or an asterisk for all of them, followed by lines that allow or refuse parts of the site. A refusal of the whole site under the asterisk is the one to look for, because it applies to every operator at once and is sometimes left behind from the weeks when a new site was being built. A refusal under a named crawler is narrower and may be deliberate. If the file does not exist, or names nothing, nothing is being refused there.
Note who put each line there before changing it. A firm may have refused a training crawler on purpose, perhaps after advice about client confidentiality, and undoing that to chase visibility would trade one concern for another. The decision belongs to the partners, and the check only tells them what the file currently says. Anthropic notes that a refusal must be made on each subdomain separately, so a firm with a separate booking or client-portal address should look at that file too.
Setting the ten-minute check against the rest of the year
Run the four links once now, then again after every change to the site. Two events undo a working chain without anyone intending it: a redesign that moves pages to new addresses, and a change of hosting account. Neither announces itself. A diary note twice a year costs less than discovering a gap after a quarter of quiet enquiries, and the written record from the first check shows what changed. For a practice with a single partner handling the website, that note is the whole process.
What the check cannot tell you
No check shows that an assistant will name the firm, because that decision belongs to the operator. All four links can hold and the firm can still go unmentioned. The checks remove avoidable reasons for absence, and that is their whole value. Anyone who offers a guaranteed placement is claiming something the operators do not publish a route to, and a sceptical partner is right to ask for the method in writing. The wider question of what such work should cost is a separate decision, and a fair price attaches to the ordinary technical items rather than to the outcome.
Who in the firm should hold the answer to link four
Link four is a writing task for the person who knows the answer, not for a supplier. The match depends on whether the page states, in plain sentences, what the firm does, for whom and where. A recruitment agency weighing candidate and client pages faces the same discipline, set out in whether a recruitment agency needs marketing or better outreach. For the sector pages we maintain, see the firms we introduce, and for how we introduce one specialist and how the fee works, read how it works.
When not to hire visibility help yet
Do not hire anyone until the firm has run the four checks and written the results down. If link one or two is broken, the fix is ordinary technical work that the existing web supplier may do in an afternoon. If link three is broken, the page needs rewriting, which no tool replaces. A supplier engaged before that will bill for finding what the partner could have found, and the invoice will read as strategy.
Frequently asked questions
What UK firms ask about being cited by an assistant.
What does an AI assistant need before it can cite a UK firm?
Four things. The operator's crawler must be allowed to read the firm's site, the page must be in that operator's index, the answer must sit on the page as text, and the page must answer the question as the client asked it. A UK practice can check the first three itself. The fourth is a judgement about the writing.
Does Google need special markup for a firm to appear in its AI answers?
Google says not. Its guidance on AI features states that a page needs to be indexed and eligible to appear in Search with a snippet, and that there are no additional technical requirements. A firm already visible in ordinary results has met the entry condition. That makes it a candidate for AI answers, not a guaranteed citation.
Is blocking one AI crawler the same as blocking them all?
No. Each major operator publishes separate crawlers with separate purposes, and every one can be allowed or refused on its own line in robots.txt. A firm that refused a training crawler may still be readable by a search crawler, and the reverse is also possible. Read the file for each operator rather than assuming.
Can anyone promise that an assistant will name my practice?
No. The operator decides what its assistant says, and none of them publishes a way to buy a place in an answer. A supplier can fix permission, indexing and the way the page is written. Nobody outside the operator can promise the outcome, and a supplier who does has told you something about their method.
Who should check this first, the firm or an agency?
The firm, for ten minutes. The checks need no technical help, and doing them first means any supplier is briefed on a specific broken link instead of a vague wish to be found. If all four links hold and the firm is still not named, that is when specialist help is worth discussing.
Sources and useful reading
- Google Search Central, AI features and your website, for the statement that a page must be indexed and eligible to appear in Search with a snippet.
- OpenAI, overview of OpenAI crawlers, for the separate crawlers for training and for search.
- Anthropic, about Claude's web crawler, for ClaudeBot, Claude-User and Claude-SearchBot and their purposes.
- Perplexity, crawler documentation, for PerplexityBot and the separate agent for user questions.
This article is commercial decision support and is not legal or technical advice. It makes no claim about what any assistant will say about any firm. The practice and its pages are invented to show the method. The operator pages are summarised from search results, not from a full reading of each page, so read the current text before relying on it. No statistic is quoted. Sources were located on 21 October 2026.