Pay-per-Crawl for Next.js · v0.1.2 · free, open sourceCharge AI crawlers on a Next.js site
Edge-runtime middleware for Next 14, 15 and 16. Humans and verified search engines read free; AI crawlers get a 402 and pay per page in USDC on Base, straight to your wallet.
Works with: Next.js 14 / 15 middleware, Next 16 proxy, Vercel or self-hosted
Install
npm install @forgemeshlabs/pay-per-crawl-next
Add the gate
import { createPayPerCrawl } from "@forgemeshlabs/pay-per-crawl-next";
const ppc = createPayPerCrawl({ siteName: "My Site" }); // PAY_TO, PRICE_USD, MODE come from env; options win
export const middleware = ppc.middleware;
// Next needs config as a literal in this file; it cannot be re-exported.
export const config = { matcher: ["/((?!_next/static|_next/image|favicon.ico).*)"] };import { createPayPerCrawl } from "@forgemeshlabs/pay-per-crawl-next";
const ppc = createPayPerCrawl({ siteName: "My Site" });
export const proxy = ppc.proxy;
export const config = { matcher: ["/((?!_next/static|_next/image|favicon.ico).*)"] };Turn it on
- 1Install the package and add the middleware file at the project root.
- 2Set PAY_TO and PRICE_USD in your environment. Leave MODE unset, which means monitor.
- 3Deploy and watch the function logs: each bot is logged with its class and what it would have paid.
- 4Set MODE=gate and redeploy to start charging.
Settings
| PAY_TO | Your 0x wallet on Base. Required to charge. |
|---|---|
| PRICE_USD | Price per page fetch in USDC. Default 0.01, minimum 0.005. |
| MODE | monitor (default) logs what bots would have paid; gate answers 402. |
| FACILITATOR_URL | Any x402 facilitator that settles on Base. Default PayAI, no API key. |
| CLASS_POLICY | JSON per class: search, social, monitor, seo, ai, automation, bot → free, paid or block. |
| BOT_OVERRIDES | JSON per crawler name, wins over the class: {"GPTBot":"paid","Bytespider":"block"}. |
| ALLOW_PATHS | Comma list of path prefixes that are never gated. |
| TRUST_HEADER | Header carrying the real client IP behind your proxy. Right-most entry is used. |
| MPP_SECRET | Long random string signing MPP challenges. Set it so challenges survive restarts. |
| PRO_KEY | Optional Pay-per-Crawl Pro license for the hosted dashboard. |
Things to know
- On Vercel the platform sets X-Forwarded-For; the gate reads the right-most entry. If your server is exposed directly, pass trustHeader: "" so a client cannot forge an IP.
- The replay lock and the reverse-DNS cache are per-process memory. Pass a store (any object with get, put, delete) to share them across instances. The on-chain nonce is the hard guard either way.
- The middleware also serves /.well-known/x402.json, /llms.txt and /agents.txt. Route handlers for them are exported if you prefer.
- Behind Cloudflare, set AI Crawl Control and any "Block AI bots" rule to Allow for the crawlers you want to charge. A crawler blocked at the edge never reaches the gate and never sees the 402.
- Humans never see a 402. Search engines are verified by reverse DNS; a request that only claims to be Googlebot is charged.
- Start in monitor mode. It logs every bot with what it would have paid and never blocks. Switch to gate when the numbers make sense.
Proof it settles
Real settlements on Base mainnet through the PayAI facilitator, $0.01 each, October 2026:
- x402 v1 (X-PAYMENT, stock x402-fetch 1.x)0x5eb3ae89264f754793de655ade581cf9e4b885946d415d1d29d8d9e079077e7c
- x402 v2 (PAYMENT-SIGNATURE, @x402/core)0x6f6e08909ef5e70e023649501721536e5e619a1008e2446f8a02830fdcc066c1