Findings and fixes
Every diagnostic Shruwd runs, grouped by family, with what triggers each one and what it asks you to do.
Every diagnostic Shruwd runs. For how findings behave — states, confidence, rechecks — see How findings work.
Reading an entry
Each rule carries a confidence and a severity, and both change how you should treat it.
| Confidence | |
|---|---|
observed | A first-party fact, from your own logs or a live fetch |
inferred | Not directly observed, but every link in the chain is in the evidence |
heuristic | A correlation against the pages being cited. A lead, not a fact |
Which findings come first
Families are not equal, and one can make another pointless.
An open access finding on a URL hides the structure and citation findings for that same URL. Recommending schema markup on a page that returns 403 to an AI crawler wastes your afternoon — fix the access problem and the rest reappear, re-evaluated.
Findings are also capped at five new per brand per week. A list of forty gets nothing done.
Where each family applies
Prompt intent decides which diagnostics run.
| Family | informational | comparison | commercial | navigational | problem |
|---|---|---|---|---|---|
| Access | ✓ | ✓ | ✓ | ✓ | ✓ |
| Citation | ✓ | ✓ | ✓ | — | ✓ |
| Structure | ✓ | ✓ | ✓ | — | ✓ |
Access
Whether AI crawlers can reach your pages at all. The highest-confidence family — the inputs are your own logs and live fetches, with no inference. Checked every day, on every plan.
robots.txt blocks an AI crawler
ACCESS_ROBOTS_BLOCK · observed · critical
Fires when your robots.txt disallows OAI-SearchBot, ChatGPT-User,
PerplexityBot or Google-Extended for a page that matters — one in your sitemap, one
with search impressions, or one already being cited.
What to do — the finding shows the offending directive with its line number and an exact replacement block as a diff.
Resolves when the directive is gone. Shruwd rereads robots.txt daily, and within minutes once you mark the fix applied.
Google-Extended affects Gemini grounding and AI training — not AI Overviews. These
get conflated constantly, and the right fix differs.
The server refuses a verified AI crawler
ACCESS_4XX_5XX_TO_AI_UA · observed · critical
The most valuable rule in the catalogue, and the one that needs crawler logs.
Fires when a verified AI crawler was refused (4xx or 5xx) more often than served on a page, on at least three separate days in the last 30, while a normal browser gets a 200 for the same URL. An occasional error does not count.
What to do — this is almost always bot protection nobody remembers enabling: Cloudflare Bot Fight Mode or AI Crawl Control, a WAF managed rule, rate limiting, a security plugin, or user-agent blocking at your origin. The finding identifies the likely blocker from the response signature and names the setting to change.
Which of these your logs can show depends on your log path. Only Logpush records refusals made by Cloudflare itself: the forwarder Worker runs after Cloudflare's security rules, and your server after that. So with the Worker, look for Cloudflare's own refusals in Cloudflare (Security → Analytics). A refusal Shruwd does see was made behind Cloudflare, and the finding tells you so.
Resolves when, after you mark the fix applied, that crawler is served on that path on a day it visits.
Content only renders with JavaScript
ACCESS_JS_ONLY_CONTENT · inferred · high
Fires when the raw HTML has under 500 characters of visible text, the page has search impressions, and the markup contains an empty root mount node.
What to do — server-render, statically generate or prerender that route. AI crawlers largely do not execute JavaScript, so content that only exists after hydration does not exist for them.
Resolves when the raw HTML carries more than 2,000 characters of visible text.
A page with demand is never crawled
ACCESS_NEVER_CRAWLED · observed · medium
Fires when a URL is in your sitemap, has at least 100 search impressions in 30 days, and has had zero verified AI-bot hits in that time.
Needs 30 days of uninterrupted crawler logs and a Search Console connection. It stays silent on brands with partial log coverage rather than blaming you for a gap in the data.
What to do — the page is reachable but undiscovered. Check how deep it sits from your
homepage, whether the sitemap includes it with an accurate lastmod, and whether anything
the crawlers already visit links to it.
Citation
Where answers get their sources. The highest-value family after access.
A third-party page gatekeeps the answer
CITE_THIRD_PARTY_GATEKEEPER · inferred · critical
The finding that answers "why does ChatGPT recommend them and not us?"
Fires when across a cluster of at least three related prompts, one third-party domain appears in 40% or more of the answers, you are absent from the page being cited, and at least one competitor is on it.
What to do — getting onto that specific source is usually the highest-leverage action available to you. The finding names the exact URLs, which competitors appear on them and how they are described, and the route in for that kind of source: a vendor listing submission, an editorial contact, community participation.
Rechecked after 30 days rather than 14 — third-party inclusion propagates slowly.
The wrong page gets cited
CITE_WRONG_PAGE · observed · high
Fires when you are cited for a prompt, but the cited URL does not match the prompt's intent — a blog post cited for a buying question when you have a pricing page.
What to do — the right page is under-optimised relative to the one being cited. Strengthen it against that intent, and link to it prominently from the page currently getting the citation.
A competitor owns this prompt
CITE_COMPETITOR_OWNS_PROMPT · observed · high
Fires when over 30 days one competitor's mention rate is at or above 60% while yours is at or below 15%, across at least ten responses.
What to do — the finding derives a content gap from the competitor pages being cited: what those pages cover that your target page does not.
Rechecked at nine repetitions after 30 days, so the verdict is conclusive on a small prompt subset.
Your page reads as stale
CITE_STALE_SOURCE · inferred · medium
Fires when a cited page of yours has no date signal newer than 18 months, in a category where the competitor pages being cited average under six months.
What to do — refresh the content and the date signals together.
Changing the date without changing the content is not the fix. Models extract facts, and stale facts get superseded whatever the date says.
Structure
How extractable your content is. Every rule here is heuristic — a lead worth testing,
not a fact.
No direct answer near the top
STRUCT_NO_DIRECT_ANSWER · heuristic · high
Fires when for a prompt where a competitor is cited and you are not, your most relevant page has no 40–120 word paragraph in the first 1,200 characters sitting under a heading that matches the question.
What to do — add a heading in the question's own form, and a short direct answer immediately beneath it, before any elaboration. The finding shows the passage from each cited competitor page that does answer it, side by side.
Needs at least two cited competitor pages to compare against.
Structured data the cited pages have
STRUCT_MISSING_SCHEMA · heuristic · medium
Fires when your page carries no JSON-LD of a type present on at least half of the competitor pages cited for the prompts it targets.
What to do — the finding generates a JSON-LD block populated from your page's actual content, not a template with placeholders to fill in.
Fewer concrete facts than the cited pages
STRUCT_LOW_FACT_DENSITY · heuristic · medium
Fires when your page's fact density is below 60% of the mean of the cited competitor pages. Density counts numerals, dates, percentages, currency amounts, measurement units and proper nouns per 100 words.
What to do — add specifics of the kinds those pages carry: concrete numbers, dates, named integrations, versions. Models cite pages containing extractable facts.
Never fires on pages under 300 words — that is a different problem.
Next steps
- How findings work — confidence, states, acting on one
- Rechecks — proving a fix worked
- Choosing a log path — unlocks two access findings