# Ingest not arriving

**Settings → Crawler logs** still reads *awaiting first logs* after you deployed. Work
through these in order — the first two catch most cases.

## 1. Test the endpoint and token directly

This separates configuration from your code entirely, and you can run it from anywhere.
`?validate=1` parses your sample and reports on it without writing anything.

```bash
curl -X POST "YOUR_ENDPOINT?validate=1" \
  -H "X-Shruwd-Ingest-Token: YOUR_TOKEN" \
  -H "Content-Type: application/x-ndjson" \
  -d '{"timestamp":"2026-01-01T00:00:00Z","host":"example.com","path":"/","user_agent":"GPTBot","status":200,"client_ip":"1.2.3.4"}'
```

| Response | Means |
|---|---|
| `usable: true` | Endpoint and token are both fine — the problem is in your app or platform. Go to step 2 |
| `401` | Wrong or revoked token. See below |
| `404` | Wrong brand in the endpoint URL |
| `422` | The sample can't be used. `reason` says why |

> **Warning**
>
> **Creating a token revokes the previous one.** If you minted a new token in Settings after
> setting up the drain, the one your app is using is dead. Only the newest token works.

## 2. On Vercel, check the drain

In **Team Settings → Drains**:

- The drain isn't marked as errored or paused.
- Its sources include `static`, `lambda`, `edge`, `external`, `redirect` and `firewall`.
- Its environments include `production`, and it has no sampling rule.

Then check **Team Settings → Security & Privacy → IP Address Visibility** is on. Hidden
IPs look like a working drain: logs arrive and the card reads *active*, but every line is
refused, so the Crawlers page stays empty.

## 3. Check the variables in the deployed environment

Set locally and forgotten in the host's configuration is the single most common cause.
Check the values in the environment your site actually runs in, not your local `.env`.

If you are using [server middleware](https://shruwd.io/docs/crawler-logs/server-middleware.md), it logs one
line the first time a send fails — look for `[shruwd] log drain not working:` in your
app's logs. That line names the cause.

If you switched the middleware to `useRuntimeConfig()`, confirm both halves: the keys
declared in `nuxt.config.ts` **and** the `NUXT_` prefix on the variables. Missing either
reads `undefined` silently.

## 4. Check whether the request was served from cache

This only applies to server middleware. A request your CDN answers never reaches your
origin, so the middleware never sees it.

Check `cf-cache-status` on the response:

| Value | Reached your origin? |
|---|---|
| `DYNAMIC` or `MISS` | Yes |
| `HIT` | No — the middleware could not have seen it |

If most of your pages are cached, middleware is the wrong path. Switch to the
[forwarder Worker](https://shruwd.io/docs/crawler-logs/cloudflare-worker.md) or a
[platform drain](https://shruwd.io/docs/crawler-logs/platform-log-drains.md).

> **Note**
>
> `cf-cache-status: DYNAMIC` means Cloudflare did not serve it from Cloudflare's cache. It
> does **not** guarantee the request reached your application — a second CDN behind it can
> still answer from static files.

## 5. Wait a little

Middleware batches: it sends at 20 events or 10 seconds, whichever comes first. One test
request will not appear immediately.

Crawler traffic is also genuinely intermittent. A low-traffic site may wait hours between
real AI-crawler hits.

## Other things worth knowing

**Logs arrive but the card shows gaps.** *Days with data* on the Settings card shows how
many of the last 30 days received anything. Gaps switch off the diagnostic that needs 30
uninterrupted days.

**Logs arrive but nothing is counted.** Check the **Unverified** column on the Crawlers
page. Hits from IPs outside the vendor's published ranges are shown but never counted —
which is correct, and still proves your pipe works.

**It stopped partway through the month.** Check **Lines this month** against your plan's
allowance. Past the cap, lines are discarded until the month turns. The Crawlers page says
so rather than showing a flat chart.

**A `413` response.** Your batches are over the size cap. On Cloudflare Logpush, set
`max_upload_bytes` to `5000000`.

## Next steps

- [Choosing a log path](https://shruwd.io/docs/crawler-logs/choosing-a-log-path.md) — if you picked the wrong one
- [Crawlers tracked](https://shruwd.io/docs/crawler-logs/crawlers-tracked.md) — how verification works