UrsaKit cover explaining how HTTP 402 is now used to charge AI crawlers and agents

HTTP status codes age slowly. 200 and 404 have meant the same thing for decades, and most of the registry is stable enough that developers stop thinking about it. Then there is 402.

402 Payment Required has existed in the HTTP specifications since 1997, listed as reserved for future use, and for almost all of that time nothing used it. In 2025 that changed from two directions at once: Cloudflare started returning 402 to charge AI crawlers for content, and the x402 standard built an entire machine-payment protocol on top of the same code.

01Why HTTP 402 Sat Unused for 25 Years

The original HTTP/1.1 specification reserved 402 without defining a payload. A status code with no agreed body format is a status code no client can act on: a browser could not know where to send money, and a proxy could not know whether to retry.

That gap mattered less than it sounds, because online payments grew up outside HTTP. Checkout pages, card forms, redirects to a payment provider and webhook callbacks all sit above the protocol. The request that arrives at your API is already paid for, so the only missing piece was a way to charge machines that talk directly to an API without a checkout page.

402 is a standard code, not a hack.

Both the pay-per-crawl and x402 designs keep the HTTP semantics intact: the resource exists, the client is understood, and the server simply refuses to deliver content until payment is settled.

02How x402 Turns 402 Into a Payment Handshake

x402 is an open standard, published under Apache-2.0, for charging for API calls and content directly over HTTP. It has no accounts, sessions or credential management; the payment instruction travels in the response headers.

The documented flow is five steps:

  1. The client requests a resource, for example GET /weather.
  2. If payment is required, the server answers 402 Payment Required with payment instructions attached.
  3. The client prepares a payment payload and retries the request.
  4. The server verifies and settles the payment, either itself or through an x402 facilitator.
  5. Once the payment is valid, the server returns the resource.

In x402's documented v2 headers, that handshake uses three names:

HeaderDirectionWhat it carries
PAYMENT-REQUIREDServer to clientBase64-encoded PaymentRequired object with schemes, price, network and destination
PAYMENT-SIGNATUREClient to serverBase64-encoded PaymentPayload proving the client authorized payment
PAYMENT-RESPONSEServer to clientBase64-encoded settlement result, success or failure

Every header value is a Base64-encoded JSON string, which keeps the payload safe from header-character rules and readable by any HTTP client.

# First request: no payment, so the server answers 402
curl -i https://api.example.com/weather

# What came back
HTTP/1.1 402 Payment Required
PAYMENT-REQUIRED: eyJhY2NlcHRzIjpbXX0=
content-length: 0

The important design decision is that the payment happens inside the normal request loop. Nothing about the URL changes, no redirect to a hosted checkout appears, and an autonomous agent can complete the whole exchange without a human.

03What Cloudflare's Pay Per Crawl Actually Returns

Cloudflare's pay-per-crawl, introduced in July 2025, applies the same code to a different problem: content owners who want to be paid when AI crawlers read their sites.

The published behaviour is straightforward. Each crawler request gets one of three outcomes chosen by the publisher: allow for free, charge at a domain-wide price, or block entirely. When the decision is charge and no payment accompanies the request, Cloudflare returns 402 Payment Required with a crawler-price header that states the price.

Two details make it more than a paywall:

  • Cloudflare acts as the merchant of record, so the publisher is not collecting individual micropayments.
  • Crawlers identify themselves with signed requests using Web Bot Auth, so a random client cannot claim a crawler's identity to negotiate a price.

In other words, 402 is used to advertise a price to a machine that might, in the future, have a billing relationship with the operator. That is a very different job from the browser-facing paywall that 401 and 403 normally handle.

04What a 402 Response Should Contain

Because the original specification never defined a body, every implementation invents its own detail. If you are adding 402 to an API, include enough for a client to act without guessing:

  • The price and the unit. Per request, per thousand requests, or per crawl — say it explicitly.
  • Where and how to pay. A payment endpoint, a network and destination address, or a scheme name.
  • How to prove payment. The header the client should send on retry, and its exact format.
  • How long the offer stands. A price with no expiry is a price that cannot be cached safely.
  • A machine-readable body. JSON is easier for agents than prose, and it keeps the response useful to both a human and a script.
{
  "error": "payment_required",
  "price": { "amount": "0.002", "currency": "USDC", "per": "request" },
  "accepts": ["exact"],
  "retry_header": "PAYMENT-SIGNATURE",
  "expires_in": 300
}

05402 vs 401, 403 and 429

Paid endpoints are new enough that teams reach for the wrong code. The four candidates are easy to separate once you name what is missing.

CodeWhat is missingWho can fix it
401 UnauthorizedValid credentialsThe caller, by authenticating
403 ForbiddenPermission, regardless of credentialsThe resource owner, by granting access
402 Payment RequiredPaymentThe caller, by paying
429 Too Many RequestsPatience or a higher quotaThe caller, by slowing down or upgrading

Two rules keep this clean. Do not return 402 for humans who simply have not logged in — that is 401. And do not return 402 for a rate limit that the caller could fix by waiting, because Retry-After on a 429 already expresses that.

402 tells a client to pay, not to retry blindly.

If your API returns 402 without instructions, well-behaved clients will treat it as a generic 4xx and stop. The value is in the headers and the body, not the number alone.

06Implementing 402 Without Breaking Clients

Adding a new status code is a compatibility decision, so treat it like one:

  1. Keep the code for machine clients. For browser traffic, redirect to a payment page or return 401 with a clear message; unknown 4xx codes confuse more users than they help.
  2. Make the first 402 cost nothing. Never charge or consume quota for the request that carries the price. Otherwise a retry loop can bill a client for requests it never received.
  3. Make the retry idempotent. A client that pays and retries must not be charged twice if the first retry is lost in transit.
  4. Log 402 separately from 4xx errors. A paid endpoint answering 402 is healthy; a 404 is not. If both are counted as "client errors" on the same dashboard, the signal disappears.
  5. Publish the header contract. Agents cannot guess PAYMENT-SIGNATURE, PAYMENT-REQUIRED or crawler-price; document them or nothing will connect.

07What This Means for Publishers and API Owners

For the first time in two decades, the least-used code in the registry has a job. That job is narrow and specific: it marks the point where an HTTP request turns into a transaction, and it lets a machine complete that transaction without a human in the loop.

Whether you are running a paid API or watching AI crawlers read your content, the practical step is the same. Check what your server returns today, decide what a client should do with each code, and make sure the response tells it what to do next.

Frequently asked questions

What does HTTP 402 Payment Required mean?

It means the server will not serve the resource until the client pays. The code has been reserved for future use in the HTTP specifications since 1997, which is why browsers never show it.

Is HTTP 402 an error on my side?

It is a 4xx client-error class code, but not necessarily a bug. If an API you call returns 402, the request reached a paid endpoint and your client needs a payment step before retrying.

What is x402?

x402 is an open standard that uses HTTP 402 to let services charge for API calls and content. A server answers without payment with 402 plus payment instructions, the client pays, then retries with proof of payment.

Which headers does x402 use?

Its documented v2 flow uses three: PAYMENT-REQUIRED on the 402 response, PAYMENT-SIGNATURE on the retry, and PAYMENT-RESPONSE after settlement. Each header carries a Base64-encoded JSON object.

How does Cloudflare pay-per-crawl differ?

Cloudflare acts as the merchant of record and returns 402 with a crawler-price header to crawlers that have not paid. Publishers choose allow, charge, or block per crawler, and crawlers identify themselves with signed requests.

Should I return 402 for a normal paywall?

For human browsers, 401 or 403 with a payment page is still the safer choice, because most browsers, crawlers and proxies treat 402 as an unknown 4xx. Reserve 402 for machine clients you expect to read the headers and retry.

Back to articles

Related articles

Related tools