# Cloudflare Logpush

A Logpush job with an HTTP destination sends your zone's request logs straight to
Shruwd. No code, and it captures everything the edge serves.

## Prerequisites

- **Permissions:** admin or owner.
- A **Cloudflare Enterprise** zone. Logpush is not available on Free, Pro or Business —
  on those plans use the [forwarder Worker](https://shruwd.io/docs/crawler-logs/cloudflare-worker.md)
  instead.

## 1. Get the destination URL

1. Open the brand's **Settings → Crawler logs**.
2. Select **Create token**.
3. Copy the **Logpush destination URL**. It already has the token embedded as a
   `header_Authorization` parameter, which Logpush turns into a request header on every
   upload.

> **Warning**
>
> The URL contains the token and is shown once. Creating a new token revokes the old one
> and will stop a job that is already running.

## 2. Create the job

In the Cloudflare dashboard, go to your zone → **Analytics & Logs → Logpush → Create a
Logpush job**.

| Step | Value |
|---|---|
| Destination | **HTTP destination** |
| HTTP endpoint | the destination URL from step 1, verbatim |
| Dataset | **HTTP requests** |
| If logs match | `ClientRequestHost` equals your domain, **or** `ClientRequestHost` equals it with `www.` in front |
| Fields | `EdgeStartTimestamp`, `ClientRequestHost`, `ClientRequestPath`, `ClientRequestUserAgent`, `EdgeResponseStatus`, `ClientIP`, `ClientRequestMethod`, `EdgeResponseBytes` |
| Advanced → timestamp format | Any. RFC3339, Unix and UnixNano are all read |
| Advanced → sampling | 100% |

On **Submit**, Cloudflare uploads a small gzipped test file and expects a `2xx`. Shruwd
answers it and does not count it toward your usage.

If job creation fails with `error validating destination`, the token in the URL is wrong
or has been revoked.

## 3. On a busy zone, cap the batch size

Shruwd reads up to 48 MB decoded per upload and drops the remainder of anything larger.
On a high-traffic zone, create the job through the API with explicit limits instead.

```bash
curl -X POST "https://api.cloudflare.com/client/v4/zones/YOUR_ZONE_ID/logpush/jobs" \
  -H "Authorization: Bearer YOUR_CLOUDFLARE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "shruwd-crawler-logs",
    "dataset": "http_requests",
    "destination_conf": "YOUR_DESTINATION_URL",
    "enabled": true,
    "filter": "{\"where\":{\"or\":[{\"key\":\"ClientRequestHost\",\"operator\":\"eq\",\"value\":\"example.com\"},{\"key\":\"ClientRequestHost\",\"operator\":\"eq\",\"value\":\"www.example.com\"}]}}",
    "output_options": {
      "field_names": ["EdgeStartTimestamp","ClientRequestHost","ClientRequestPath","ClientRequestUserAgent","EdgeResponseStatus","ClientIP","ClientRequestMethod","EdgeResponseBytes"],
      "timestamp_format": "rfc3339"
    },
    "max_upload_bytes": 5000000,
    "max_upload_records": 10000
  }'
```

## 4. Verify

Within a few minutes of traffic, **Settings → Crawler logs** moves from *awaiting first
logs* to *active*.

A hit from an IP outside the crawler vendor's published ranges is recorded as
**unverified** and excluded from headline metrics. That is correct behaviour, and it
still proves the pipe is working.

## Next steps

- [Crawlers tracked](https://shruwd.io/docs/crawler-logs/crawlers-tracked.md) — how verification works
- [Ingest not arriving](https://shruwd.io/docs/crawler-logs/ingest-not-arriving.md) — if nothing shows up