# Findings reference

Every diagnostic Shruwd runs. For how findings behave — states, confidence, rechecks —
see [How findings work](https://shruwd.io/docs/findings/how-findings-work.md).

## Reading an entry

Each rule carries a **confidence** and a **severity**, and both change how you should
treat it.

| Confidence | |
|---|---|
| `observed` | A first-party fact, from your own logs or a live fetch |
| `inferred` | Not directly observed, but every link in the chain is in the evidence |
| `heuristic` | A correlation against the pages being cited. A lead, not a fact |

## Which findings come first

Families are not equal, and one can make another pointless.

**An open access finding on a URL hides the structure and citation findings for that same
URL.** Recommending schema markup on a page that returns 403 to an AI crawler wastes your
afternoon — fix the access problem and the rest reappear, re-evaluated.

Findings are also capped at five new per brand per week. A list of forty gets nothing done.

## Where each family applies

Prompt intent decides which diagnostics run.

| Family | informational | comparison | commercial | navigational | problem |
|---|:--:|:--:|:--:|:--:|:--:|
| Access | ✓ | ✓ | ✓ | ✓ | ✓ |
| Citation | ✓ | ✓ | ✓ | — | ✓ |
| Structure | ✓ | ✓ | ✓ | — | ✓ |

---

## Access

Whether AI crawlers can reach your pages at all. The highest-confidence family — the
inputs are your own logs and live fetches, with no inference. Checked every day, on every
plan.

### robots.txt blocks an AI crawler

`ACCESS_ROBOTS_BLOCK` · observed · **critical**

**Fires when** your `robots.txt` disallows `OAI-SearchBot`, `ChatGPT-User`,
`PerplexityBot` or `Google-Extended` for a page that matters — one in your sitemap, one
with search impressions, or one already being cited.

**What to do** — the finding shows the offending directive with its line number and an
exact replacement block as a diff.

**Resolves when** the directive is gone. Shruwd rereads robots.txt daily, and within
minutes once you mark the fix applied.

> **Note**
>
> `Google-Extended` affects Gemini grounding and AI training — **not** AI Overviews. These
> get conflated constantly, and the right fix differs.

### The server refuses a verified AI crawler

`ACCESS_4XX_5XX_TO_AI_UA` · observed · **critical**

The most valuable rule in the catalogue, and the one that needs
[crawler logs](https://shruwd.io/docs/crawler-logs/choosing-a-log-path.md).

**Fires when** a verified AI crawler was refused (4xx or 5xx) more often than served on a
page, on at least three separate days in the last 30, while a normal browser gets a 200
for the same URL. An occasional error does not count.

**What to do** — this is almost always bot protection nobody remembers enabling:
Cloudflare Bot Fight Mode or AI Crawl Control, a WAF managed rule, rate limiting, a
security plugin, or user-agent blocking at your origin. The finding identifies the likely
blocker from the response signature and names the setting to change.

Which of these your logs can show depends on your
[log path](https://shruwd.io/docs/crawler-logs/choosing-a-log-path.md). Only Logpush records refusals made by
Cloudflare itself: the forwarder Worker runs after Cloudflare's security rules, and your
server after that. So with the Worker, look for Cloudflare's own refusals in Cloudflare
(**Security → Analytics**). A refusal Shruwd does see was made behind Cloudflare, and the
finding tells you so.

**Resolves when**, after you mark the fix applied, that crawler is served on that path
on a day it visits.

### Content only renders with JavaScript

`ACCESS_JS_ONLY_CONTENT` · inferred · **high**

**Fires when** the raw HTML has under 500 characters of visible text, the page has search
impressions, and the markup contains an empty root mount node.

**What to do** — server-render, statically generate or prerender that route. AI crawlers
largely do not execute JavaScript, so content that only exists after hydration does not
exist for them.

**Resolves when** the raw HTML carries more than 2,000 characters of visible text.

### A page with demand is never crawled

`ACCESS_NEVER_CRAWLED` · observed · **medium**

**Fires when** a URL is in your sitemap, has at least 100 search impressions in 30 days,
and has had **zero** verified AI-bot hits in that time.

Needs 30 days of uninterrupted [crawler logs](https://shruwd.io/docs/crawler-logs/choosing-a-log-path.md) and
a [Search Console connection](https://shruwd.io/docs/tracking/connecting-search-console.md). It stays silent
on brands with partial log coverage rather than blaming you for a gap in the data.

**What to do** — the page is reachable but undiscovered. Check how deep it sits from your
homepage, whether the sitemap includes it with an accurate `lastmod`, and whether anything
the crawlers already visit links to it.

---

## Citation

Where answers get their sources. The highest-value family after access.

### A third-party page gatekeeps the answer

`CITE_THIRD_PARTY_GATEKEEPER` · inferred · **critical**

The finding that answers "why does ChatGPT recommend them and not us?"

**Fires when** across a cluster of at least three related prompts, one third-party domain
appears in 40% or more of the answers, you are absent from the page being cited, and at
least one competitor is on it.

**What to do** — getting onto that specific source is usually the highest-leverage action
available to you. The finding names the exact URLs, which competitors appear on them and
how they are described, and the route in for that kind of source: a vendor listing
submission, an editorial contact, community participation.

Rechecked after 30 days rather than 14 — third-party inclusion propagates slowly.

### The wrong page gets cited

`CITE_WRONG_PAGE` · observed · **high**

**Fires when** you are cited for a prompt, but the cited URL does not match the prompt's
intent — a blog post cited for a buying question when you have a pricing page.

**What to do** — the right page is under-optimised relative to the one being cited.
Strengthen it against that intent, and link to it prominently from the page currently
getting the citation.

### A competitor owns this prompt

`CITE_COMPETITOR_OWNS_PROMPT` · observed · **high**

**Fires when** over 30 days one competitor's mention rate is at or above 60% while yours
is at or below 15%, across at least ten responses.

**What to do** — the finding derives a content gap from the competitor pages being cited:
what those pages cover that your target page does not.

Rechecked at nine repetitions after 30 days, so the verdict is conclusive on a small
prompt subset.

### Your page reads as stale

`CITE_STALE_SOURCE` · inferred · **medium**

**Fires when** a cited page of yours has no date signal newer than 18 months, in a
category where the competitor pages being cited average under six months.

**What to do** — refresh the content and the date signals together.

> **Warning**
>
> Changing the date without changing the content is not the fix. Models extract facts, and
> stale facts get superseded whatever the date says.

---

## Structure

How extractable your content is. Every rule here is `heuristic` — a lead worth testing,
not a fact.

### No direct answer near the top

`STRUCT_NO_DIRECT_ANSWER` · heuristic · **high**

**Fires when** for a prompt where a competitor is cited and you are not, your most
relevant page has no 40–120 word paragraph in the first 1,200 characters sitting under a
heading that matches the question.

**What to do** — add a heading in the question's own form, and a short direct answer
immediately beneath it, before any elaboration. The finding shows the passage from each
cited competitor page that does answer it, side by side.

Needs at least two cited competitor pages to compare against.

### Structured data the cited pages have

`STRUCT_MISSING_SCHEMA` · heuristic · **medium**

**Fires when** your page carries no JSON-LD of a type present on at least half of the
competitor pages cited for the prompts it targets.

**What to do** — the finding generates a JSON-LD block populated from your page's actual
content, not a template with placeholders to fill in.

### Fewer concrete facts than the cited pages

`STRUCT_LOW_FACT_DENSITY` · heuristic · **medium**

**Fires when** your page's fact density is below 60% of the mean of the cited competitor
pages. Density counts numerals, dates, percentages, currency amounts, measurement units
and proper nouns per 100 words.

**What to do** — add specifics of the kinds those pages carry: concrete numbers, dates,
named integrations, versions. Models cite pages containing extractable facts.

Never fires on pages under 300 words — that is a different problem.

---

## Next steps

- [How findings work](https://shruwd.io/docs/findings/how-findings-work.md) — confidence, states, acting on one
- [Rechecks](https://shruwd.io/docs/findings/rechecks.md) — proving a fix worked
- [Choosing a log path](https://shruwd.io/docs/crawler-logs/choosing-a-log-path.md) — unlocks two access findings