Pay-per-Crawl for Python · v0.1.2 · free, open sourceCharge AI crawlers in FastAPI, Flask or Django
One package, three adapters: ASGI for FastAPI and Starlette, WSGI for Flask and anything else, and a Django middleware. Python 3.10+, one dependency.
Works with: FastAPI, Starlette, Flask, any WSGI app, Django
Install
pip install forgemesh-pay-per-crawl
Add the gate
from forgemesh_pay_per_crawl import Config
from forgemesh_pay_per_crawl.asgi import PayPerCrawlMiddleware
app.add_middleware(PayPerCrawlMiddleware,
config=Config(pay_to="0xYourWalletOnBase", mode="gate", price_usd=0.01, base_url="https://example.com"))from forgemesh_pay_per_crawl import Config
from forgemesh_pay_per_crawl.wsgi import PayPerCrawlWSGI
app.wsgi_app = PayPerCrawlWSGI(app.wsgi_app,
Config(pay_to="0xYourWalletOnBase", mode="gate", price_usd=0.01, base_url="https://example.com"))MIDDLEWARE = ["forgemesh_pay_per_crawl.django.PayPerCrawlMiddleware", *MIDDLEWARE] # near the top
PAY_PER_CRAWL = {"pay_to": "0xYourWalletOnBase", "mode": "gate", "price_usd": 0.01, "base_url": "https://example.com"}Turn it on
- 1pip install the package and add the adapter for your framework.
- 2Start with mode="monitor" and pass base_url if you are behind a proxy.
- 3Read the on_event callback or your logs for a day, then switch to mode="gate".
Settings
| pay_to | Your 0x wallet on Base. Required in gate mode. |
|---|---|
| mode | "monitor" (default) records only; "gate" answers 402. |
| price_usd | Price per page fetch, default 0.01, minimum 0.005. |
| facilitator_url | Any x402 facilitator that settles on Base. Default PayAI, no key. |
| class_policy / bot_overrides | Per class and per crawler: "free", "paid" or "block". |
| allow_paths | Path prefixes never gated. robots.txt, sitemaps, llms.txt, agents.txt and /.well-known/ always are. |
| base_url | Your public origin. Set it behind a proxy or load balancer. |
| trust_proxy_header | Header with the real client IP ("CF-Connecting-IP", "X-Forwarded-For"). Right-most entry used. |
| mpp_secret | Stable HMAC key for MPP challenge ids. Set it when you run more than one worker. |
Things to know
- With several worker processes, pass lock=FileNonceLock("/var/tmp/fmppc") so a payment cannot be replayed on another worker, and set a stable mpp_secret.
- Gate.handle is blocking because it can call the facilitator. In async code run it in a thread, as the ASGI middleware does.
- Behind Cloudflare, set AI Crawl Control and any "Block AI bots" rule to Allow for the crawlers you want to charge. A crawler blocked at the edge never reaches the gate and never sees the 402.
- Humans never see a 402. Search engines are verified by reverse DNS; a request that only claims to be Googlebot is charged.
- Start in monitor mode. It logs every bot with what it would have paid and never blocks. Switch to gate when the numbers make sense.
Proof it settles
Real settlements on Base mainnet through the PayAI facilitator, $0.01 each, October 2026:
- x402 v10x49f0f05b3509545e5134aa6cf091ff7eab4e4f1e1b9aef9bf28280c231e2a945
- x402 v20x15d95ad90c9530e4816016f65681ddd4a79730373634c991c57afa2b88a8b44b